TechSpot

File sync + online DB buying advice

By FAll3N
Feb 3, 2012
Post New Reply
  1. Ok, I am about to setup a server used for file syncing and an online db (like SQLite or something…) anyway, I also wanna have Fedora running on it..

    I guess I can just get any server and install what I want + make it do what I want it to do, but what server(s) and/or system specs should I be specifically looking at for doing those things better than other servers?
     
  2. Leeky

    Leeky TS Evangelist Posts: 3,797   +116

    Depends. Do you actually need a server?

    Are we talking about a dedicated server in a datacenter, or are you building a server at home/work and using it connected to the internet there?

    You need to tell us what you want to do with the server. If its a dedicated server, what do you plan to do with it? Monthly bandwidth, unique visitor numbers, number of sites, etc.

    For homeserver, again, what is your intended usage. Your current post is far too vague to give any realistic answer.
     
  3. jobeard

    jobeard TS Ambassador Posts: 9,311   +617

    Asking: What data are you 'syncing' and what other devices or systems are importing this data?

    Is the sync totally contained inside your Lan or needs access from the Internet?

    Can you program SQL and / or CGI to implement this yourself?
     
  4. FAll3N

    FAll3N TS Rookie Topic Starter

    I am syncing a folder and it's contents (probably HTML files and other folders)

    This specific project is contained inside my LAN, however the db would need to be accessible to other machines.. so I guess, the answer to your question would be partially.

    I am learning SQL, am fair (not great) at CGI (but I am getting better) but in terms of implementing this myself... I will be able to maintain it myself, but to get started, I do need to ask for help, which is why I ask you.
     
  5. jobeard

    jobeard TS Ambassador Posts: 9,311   +617

    hmm; Syncing files/folders is a relatively easy job and is much like an intelegent copy;
    copy A->B if/only if date of A younger than that B.

    Second consideration is 'Which system is the "master" and considered to always be the current copy'? With a single master, it's a snap, but if non-master version get modified independently, then you can run in circles a->b; b'->c and b'->back to a.

    Third issue is the DB representation. Take for example a graphic file of which there are several formats; eg JPG, GIF, PNG. Getting a DB field to hold an arbitrary object is usually done with a field type BLOB which is just a flock of unstructured bits.
    Inserting JPG->Blob(graphic) is fine, but how then do you store the GIF & PNG?
    One approach might be a table of two fields
    * objectType
    * objectData

    insert into objectName=graphicFileName,
    objectType = "JPG",
    objectData(x) table SYNCDATA;

    defining objectType to be one of {JPG,GIF,PNG,HTML,CSS,JS} would then allow
    SYNCDATA to hold all types of webpage contents.

    Retreival then would have access to the file type in the objectType field,
    the object itself, and the original fileName from objectName.

    As a webpage can be a collection of files sent to the browser;
    < script type=javascript src=filename > < / script >
    < style type=text/css src=cssfilename > < / style>

    you may need a 'parentObjectName' to relate where the pieces belong

    and you have a BIG problem parsing and reassembling html where tags
    are embedded within the file itself rather that neatly externalized with src=x or href=y
    Code:
    <html><head>
        <script LANGUAGE="JavaScript">
            <!-- Begin JavaScript ---- scroller for status bar
            function scrollit(seed) { ....
            }
       </script>
       <script src=emv.js ></script>
    </head>
    <body>
    </body></html>
    For this, IGNORE the inline stuff and just save the HTML which implicitly saves the embeds.

    Having come this far, it is starting to look like a Content Management System(CMS) - -
    Is that your intent?
     
  6. FAll3N

    FAll3N TS Rookie Topic Starter

    Yes, sounds to me like a CMS (but one running some linux distro is important to me, cuz I have been learning at school on Linux. But, someone said CentOS would be better to use for my situation, so I can do that too...)

    I was thinking of using '$ rysnc' for the file syncing... I believe that is one of the very common uses of rsync, and it does exactly what you said:
    however, I am not set on using rsync (meaning if there is another solution that is much better, or rsync would not work or something, I have no problem with switching to something else.

    if by
    you mean is it a fileserver where several people can make changes to the files, with logins and versioning (like for group programs or something) the answer is no.

    This particular server is mostly a backup.. I primarily run mac, but I have a PC laptop from my school also. As for the online db portion, I will probably just store the db(s) as files, or something else, but I'll figure that out later, when I get to it.

    For now, I just need to get the server, lol... Which server should I get for the above purposes and why?
     
  7. jobeard

    jobeard TS Ambassador Posts: 9,311   +617

    Here's a list of other Linux sync tools you might consider.

    One issue for command line tools is the passing of user/pwd as an argument which makes it visible on the
    system issuing the command and open to discovery when placed into a script file.
    An alternative is to open the connection to the target system with SSH having prestored the user's key - - here's how

    A light just came on for me; do you need to sync arbitrary files like Word, Excel, Powerpoint, Notepad - -
    OR is this a Web content project syncing website files only?
    For projects which have programmed source code, you and the team need to learn
    Concurent Versions System(CVS). Lots of project management tools and controls
    to enhance the control of the project. Here's the manpage for man cvs
    No, the concept of 'master' is a point of reference for 'who is in control'; which system
    holds the original file(s). All others are just copies - - until someone elects to change a non-master version.
    backup is backup is backup. This is a big subject by itself. I'll defer that to another discussion
    You can get almost anything, but imo the issue is about administration of the server and using server tools,
    not functions, features and that massively naive question 'which one is best'.

    If your network is homogeneous (all PCs, all Macs, ...) then stick to that platform.

    For heterogeneous networks of mixed systems, it is mixed bag of 'which problems need what skills'.
    Networking and getting r/w access can be an issue.
    Backup of multiple system platforms is a chore too.
     

Similar Topics

Add New Comment

You need to be a member to leave a comment. Join thousands of tech enthusiasts and participate.
TechSpot Account You may also...