The local server that comes with the Google App Engine SDK tries to emulate the deployment environment as much as possible. This is a very important goal and I hope to see this improve in the future. Currently, the modules that are not supported on the deployment server are deleted from the various models. Those modules that are semi-supported at monkeypatched.
The Bypassing local server restrictions on Google App Engine article by Aral Balkan, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-Noncommercial 2.0 UK: England License.
The problem is that the unsupported modules are completely deleted. Instead, they should really be renamed and kept on so that they can be re-enabled temporarily if and when you need that functionality locally.
I rant into a real-world use case for this today when building the feature to download a remote datastore backup to the local environment. I need to create folders and os.mkdir is one of the disabled modules.
I tried to find a way of doing this without having to modify the local server but couldn't find one. Instead I modified it as follows:
Line 1119:
for symbol in set(module.__dict__) - set(allowed_symbols):
if not (symbol.startswith('__') and symbol.endswith('__')):
+ module.__dict__['old_'+symbol] = module.__dict__[symbol]
del module.__dict__[symbol]
Update: In AppEngine 1.1.7, apply the patch to Line 1148 of dev_appserver.py. (You can find dev_appserver.py in /Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/ on a default install.)
Linux and Windows users: You may have to set the files in google_appengine to writable (chmod +w -R google_appengine) before you can edit it.
Windows users: You may have to set your security settings so that you can actually edit the dev_appserver.py file (Properties → Security → Users and set Full Control on).
Then, in my download handler (which only runs on the local environment), I monkeypatch mkdir back temporarily. I also add back the write mode to file:
from google.appengine.tools import dev_appserver # Add 'w' to the allowed modes OLD_ALLOWED_MODES = dev_appserver.FakeFile.ALLOWED_MODES dev_appserver.FakeFile.ALLOWED_MODES = frozenset(['r', 'rb', 'U', 'rU', 'w']) # Add mkdir back to os os.mkdir = os.old_mkdir
Finally, at the end of the method, when the download is complete, I return things to the way they were:
# Remove 'w' from the allowed modes for file. dev_appserver.FakeFile.ALLOWED_MODES = OLD_ALLOWED_MODES # Remove mkdir from os del os.mkdir
I'd love it if dev_appserver.py worked this way without the patch so I've opened Issue 616. I know this sounds like a very edge case but it's quite a central part of the backup/restore process I'm creating.
It's not a big deal if it's not patched -- it will simply mean that you'll have to apply a patch before you can use the backup/restore solution -- but it would be nice to have.
(And, in general, I feel it's a good practice to keep a copy of the old function when you monkeypatch to provide flexibility for cases just like this.)
The Bypassing local server restrictions on Google App Engine article by Aral Balkan, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-Noncommercial 2.0 UK: England License.
[...] – bookmarked by 6 members originally found by rodolfocalado on 2008-09-29 Bypassing local server restrictions on Google App Engine http://aralbalkan.com/1440 – bookmarked by 1 members originally found by cvos on 2008-09-29 [...]
Why does the patch look like it deletes one line and inserts two, of which the latter is identical to the deleted line? It makes more sense to do this:
for symbol in set(module.__dict__) - set(allowed_symbols): if not (symbol.startswith('__') and symbol.endswith('__')): + module.__dict__['old_'+symbol] = module.__dict__[symbol] del module.__dict__[symbol]Shit. The comment box messed up the whitespace, and there doesn’t seem to be a preview feature or a way to turn that off. Use View source to see what I meant.
Hey Guido,
I updated the post with your version and fixed your comment too. I should really state somewhere that people can use
in the comments.
Version 1.1.9 on a Mac it is at line 1277 here: /Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py
Instead of hacking the dev server, why not just use reload(os) to get back the deleted functions?
Hi, I patched dev_appserver.py as described but with gaebar-aep I still get the error message: AttributeError at /populate-datastore/
‘module’ object has no attribute ‘old_remove’
I’m using App Engine 1.2.0. Is it possible that your patch doesn’t work with this version of App Engine anymore?
On 1.2.0, it’s on line 1435: /Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py
1.2.2 is the same as 1.2.0
Since you wrote this the line in question seems to have moved to 1437.
And I didn’t understand the second bit from *Then, in my download handler* … I don’t have to do anything here do?
Hi aral, I’ve posted a few issues about downloading the files for gaebar…
http://github.com/aral/gaebar-gaed-skeleton/issues/#issue/1
… I don’t seem to be able to get all the bits I need… so close…
Where do I do this bit? “Then, in my download handler … etc”
in GoogleAppEngineLauncher 1.2.3 (Mac of course), it’s moved to line 1482
Now (1.2.4) at line 1504.
Fantastic job Aral!
1.2.5 line 1510
Hi, first of all, thanks for this great tool..
I’ve the same proble that it have been commented (by Stefan) before.
Then, I could not use the Gaebar yet to make backups…!
”
Hi, I patched dev_appserver.py as described but with gaebar-aep I still get the error message: AttributeError at /populate-datastore/
‘module’ object has no attribute ‘old_remove’
I’m using App Engine 1.2.0. Is it possible that your patch doesn’t work with this version of App Engine anymore?”
Somebody can help me?
Now at line 1615 in v1.3.1
Great work, Aral!
I think Leon and Stefan’s problems could be solved by commenting out ‘remove’ in _WHITE_LIST_PARTIAL_MODULES['os']
In GAELauncher (Mac) v1.3.3, the line is now at 1629
In GAELauncher (Mac) v1.3.4, the line is now at 1634