This is a quick screencast to show you the data export solution I've created for Google App Engine that lets you backup your application's datastore and restore it either locally on your development machine or on the same Google App Engine application on the deployment environment or on a different Google App Engine application (which you can use as a staging environment).
That's right, this solves one of the big criticisms I've seen levelled at Google App Engine, which is that there is no way to export your data from your applications.
With this solution, not only can you export your data but you can also easily restore it.
I am going to be sharing the solution with you in the coming days as a separate open source project in the form of a Django application that you can plug into your own applications as part of the Singularity Web Conference Open Initiatives program (stay tuned for updates on that front).
In the meanwhile, watch the screencast to see how the backup and restore solution works.
View the full-resolution screencast on screencast.com
The Screencast demonstrating the first Google App Engine data export solution (full backup and restore) article by Aral Balkan, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-Noncommercial 2.0 UK: England License.
Very nice Aral. I’ll want to use that.
I have a question about building Singularity on GAE: are you at all concerned you might hit your account limits? Considering this is a commercial application, and HOPEFULLY you’ll get tons of traffic and stuff going on for Singularity, it would be quite bad if you hit those limits and users couldn’t participate in the conference or see the information. How do you keep within your limits?
The reason I ask is because I’d like to host a commercial app on GAE, but ensure I stay under the limits or be able to buy more resources by the time I exceed the free limits.
Thanks for any feedback!
That’s very cool Aral. Congrats.
Did you take try approaches other than serializing into a .py file. Perhaps into pickles that are zipped and sliced. You dump them into memcache, and later retrieve from memcache, unzip, unpickle and store into local….
I’m not sure if that’s gonna work, just throwing ideas :)
Nice screencast Aral.
I had read your post on the Google group, and I was pretty skeptical, but this looks *really* good! Nice work–I look forward to playing around with this.
Aral,
Any idea when this will be ready for others? I could *really* use it to do some testing in my local environment using live data! Thanks for the time + effort!
Is this a dead project? I noticed the last update time is August this year …
[...] it’s time for me to go back to my Google App Engine backup solution, which broke after Google introduced artificial limits on query offsets, and update it to use [...]
Nice screencast