How do I save JSON files to off-line database?

[aheight]

Hi,

I've been working on a 3D rendering web application and am considering creating a database of JSON files. Currently I create a JSON file of data in Mathematica, then run Dropbox to load the JSON file into Dropbox and manually created the web link. I then have to manually add a url to my Javascript file to then access that file. I can envison perhaps a database with many files.

It would be nice if I could somehow completely automate this process. What if I just saved the JSON files to some off-line database that would then automatically update some sort of JSON "url" file. I could then read-in the url file with my Javascript app and then populate a menu with all the files without ever having to see or type-in a bunch of urls?

I'm new to web programming and was wondering how might someone with more experience than me implement such a process or do I have no choice other than manually type in the url's for each file?

Would a SQL database work in this case? Not familiar with SQL though but could learn.

Thanks,

 

[Pepper Mint]

If you would really want to use database then I suggest MongoDB. Its data are all JSON formatted. I personally find it would probably overkill your current problem though.

 

[aheight]

Hi PepperMint,

Why do you think this would be over-kill? I can envision a database with thousands of files. What other option do I have to automate this process? I currently have 7 files hard-coded in my app and it's a pain to update everything each time I need to add a new one.

Do you know about the on-line encyclopedia of integer sequences? Well, maybe this app could turn into something like that.

 

[QuantumQuest]

Hi,

I've been working on a 3D rendering web application and am considering creating a database of JSON files. Currently I create a JSON file of data in Mathematica, then run Dropbox to load the JSON file into Dropbox and manually created the web link. I then have to manually add a url to my Javascript file to then access that file. I can envison perhaps a database with many files.

It would be nice if I could somehow completely automate this process. What if I just saved the JSON files to some off-line database that would then automatically update some sort of JSON "url" file. I could then read-in the url file with my Javascript app and then populate a menu with all the files without ever having to see or type-in a bunch of urls?

I'm new to web programming and was wondering how might someone with more experience than me implement such a process or do I have no choice other than manually type in the url's for each file?

Would a SQL database work in this case? Not familiar with SQL though but could learn.

Thanks,

In order to solve the problem effectively, you have to think carefully about what this solution should involve (parts - processes).

Nothing is done automatically, unless it is in some way scripted to be so. In your case, there is a client side program and a database - as you describe it. How these could communicate?

 

[aheight]

Nothing is done automatically, unless it is in some way scripted to be so. In your case, there is a client side program and a database - as you describe it. How these could communicate?

Hi Quantum,

Would I save the JSON files in a database and then write a SQL query in Javascript to access the database and request the server send the file to my application?
I have never written a Web database application. Would you know of something simple and free I could experiment with for example, just save a simple "Hello World" record to some database, write a simple SQL request, retrieve the data, then print it out on the screen?

 

[QuantumQuest]

Would I save the JSON files in a database and then write a SQL query in Javascript to access the database and request the server send the file to my application?

That's why I started in #4 to describe the whole thing as a solution. In order for you to see what's involved and think about what can be done.

I have never written a Web database application. Would you know of something simple and free I could experiment with for example, just save a simple "Hello World" record to some database, write a simple SQL request, retrieve the data, then print it out on the screen?

So, the solution contains a client side app, a server side app and a DBMS (Database Management System) (SQL flavor or NoSQL).

You can start by downloading a WAMP (Windows Apache MySQL PHP) application, if you are on Windows 7+ (32 or 64 bits, two versions available). If you run some Linux distro, some LAMP stack (Linux Apache MySQL PHP) will be available through your package manager. This, will contain everything you need to develop a full web application on your machine, is open source and is very easy to tune up (in this one I linked to, there is not even need to edit the configuration files for Apache web server). I recommend this application because of the easiness to install and configure, but of course there are other options too (other servers, wamp stacks, server-scripting languages, frameworks etc. open source or not).

Having this installed, you have to learn server side development (in this case PHP). What can do PHP for you, is to be used to write scripts on the server, in order to do usual programming tasks, communicate with the database management system, handle input validation and handle requests - responses (among other things). Then, you also have to learn some basics about databases and DBMSs. If you follow my recommendation about WAMP or LAMP, then you'll have to learn about MySQL.

Now, with all these in place, you have to learn about how a full web app works i.e client side app - requests - protocols used - responses - model used for server side app, to name the most important.

There are also some hybrid JavaScript solutions (instead of a full web stack), like Node.js.

Database can be NoSQL as well. It is a more light-weight solution compared to relational database, but its appropriateness depends on the kind, number and size of files.

Last but not least you have to learn about web services. They are used for machine-to-machine communication in general, e.g. you can talk from one web server to another.

 

[aheight]

Sounds really cool and interesting. A journey begins with a first step! Thanks Quantum. I'll look into it.:)

 

[aheight]

Ok Quantum, I downloaded WAMP, logged into the PHP tutorial site:
http://www.tutorialrepublic.com/php-tutorial/php-mysql-select-query.php

then created a database, set up a table, inserted some data, then retrieved it. Here's my PHP to query the database and retrieve the data.

Don't wish to jump the gun -- will practice with the database a while with just simple data, but could you tell me if I can save JSON files to the database and retrieve them this way?

<?php
/* Attempt MySQL server connection. Assuming you are running MySQL
server with default setting (user 'root' with no password) */
$link = mysqli_connect("localhost", "root", "", "demo");

// Check connection
if($link === false){
    die("ERROR: Could not connect. " . mysqli_connect_error());
}

// Attempt select query execution
$sql = "SELECT * FROM persons";
if($result = mysqli_query($link, $sql)){
    if(mysqli_num_rows($result) > 0){
        echo "<table>";
            echo "<tr>";
                echo "<th>person_id</th>";
                echo "<th>first_name</th>";
                echo "<th>last_name</th>";
                echo "<th>email_address</th>";
            echo "</tr>";
        while($row = mysqli_fetch_array($result)){
            echo "<tr>";
                echo "<td>" . $row['person_id'] . "</td>";
                echo "<td>" . $row['first_name'] . "</td>";
                echo "<td>" . $row['last_name'] . "</td>";
                echo "<td>" . $row['email_address'] . "</td>";
            echo "</tr>";
        }
        echo "</table>";
        // Close result set
        mysqli_free_result($result);
    } else{
        echo "No records matching your query were found.";
    }
} else{
    echo "ERROR: Could not able to execute $sql. " . mysqli_error($link);
}

// Close connection
mysqli_close($link);
?>

. . . that's one small step for man . . . :)

 

[QuantumQuest]

...but could you tell me if I can save JSON files to the database and retrieve them this way?

You can save various kinds of data in the database, even binary (blob in MySQL). When you have binary files, common practice is to save the address (the URI in general) and not the file itself into the database, for various reasons. Now, json are just text files. I'll leave it to you to figure out, after some studying and experimenting, what is the better way to store and retrieve these files. Keep in mind, that what you want to accomplish, is creating each time an html file, include json files into it - and whatever other files you want to include, and send to the browser. Also, I highly recommend reading PHP manual and MySQL manual beyond tutorials. There are also great books about these. See https://www.amazon.com/PHP-Scripting-Programming-Books/&tag=pfamazon01-20.

EDIT: I also highly recommend learning XML, if you're not familiar with it. You'll need it in various cases in web development, either as pure XML or as an intermediate type format.