Calvin's

Icon

designs and hacks. people and products.

Installing pyscopg2 in virtualenv against postgresql92, instead of postgresql91

In my previous post, I explained how we can run postgresql92 and postgresql91 concurrently.  However, that post does not explain how we can ensure that the psycopg2 we install in our project’s python virtual environment will build against postgresql92 server.

When psycopg2 is installed (via `pip install psycopg2` for instance), psycopg2 depends on the presence of `pg_config` to locate the currently active postgresql version.

If we do `pg_config –sharedir`, we will see

calvin$ pg_config --sharedir
/opt/local/share/postgresql91

which tells us that if we run `pip install virtualenv`, we will essentially be building psycopg2 against postgresql91.

Fortunately for us, using macports, it’s trivial and completely painless to switch our postgresql server version.


calvin$ sudo port select --list postgresql
Password:
Available versions for postgresql:
 none
 postgresql91 (active)
 postgresql92

calvin$ sudo port select --set postgresql postgresql92
Selecting 'postgresql92' for 'postgresql' succeeded. 'postgresql92' is now active.

calvin$ sudo port select --list postgresql
Available versions for postgresql:
 none
 postgresql91
 postgresql92 (active)

Now, running `pip install psycopg2` in our python virtual environment will correctly build our psycopg2 against our postgresql 9.2 server.

calvin$ pip install psycopg2
Downloading/unpacking psycopg2
 Downloading psycopg2-2.4.5.tar.gz (719kB): 719kB downloaded
 Running setup.py egg_info for package psycopg2

 no previously-included directories found matching 'doc/src/_build'
Installing collected packages: psycopg2
 Running setup.py install for psycopg2
 building 'psycopg2._psycopg' extension
 /usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -pipe -O2 -fwrapv -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.5 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x090202 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -I. -I/opt/local/include/postgresql92 -I/opt/local/include/postgresql92/server -c psycopg/psycopgmodule.c -o build/temp.macosx-10.7-x86_64-2.7/psycopg/psycopgmodule.o
 /usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -pipe -O2 -fwrapv -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.5 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x090202 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -I. -I/opt/local/include/postgresql92 -I/opt/local/include/postgresql92/server -c psycopg/green.c -o build/temp.macosx-10.7-x86_64-2.7/psycopg/green.o
 /usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -pipe -O2 -fwrapv -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.5 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x090202 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -I. -I/opt/local/include/postgresql92 -I/opt/local/include/postgresql92/server -c psycopg/pqpath.c -o build/temp.macosx-10.7-x86_64-2.7/psycopg/pqpath.o
 /usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -pipe -O2 -fwrapv -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.5 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x090202 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -I. -I/opt/local/include/postgresql92 -I/opt/local/include/postgresql92/server -c psycopg/utils.c -o build/temp.macosx-10.7-x86_64-2.7/psycopg/utils.o
 /usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -pipe -O2 -fwrapv -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.5 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x090202 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -I. -I/opt/local/include/postgresql92 -I/opt/local/include/postgresql92/server -c psycopg/bytes_format.c -o build/temp.macosx-10.7-x86_64-2.7/psycopg/bytes_format.o

Q.E.D.

Next! :-)

geodjango: django tests break when creating new postgis test databases

The new postgis 2.0 library (easily installed via `sudo port install postgis2 +postgresql92` if you are using macports as mentioned in my previous post) helps us convert a standard postgresql database into a spatially-aware database via the simple `CREATE EXTENSION postgis;` command:

$  psql92 -p 5433 -U postgres -d your_db

psql92 (9.2.1)
Type "help" for help.
your_db=#  CREATE EXTENSION postgis;

Unfortunately, when we run our django tests, `test_your_db`, which is newly created by django’s test runner whenever the test begins, will not be postgis-enabled. This leads immediately to test failure when our code fails to recognize the Geometry fields in our project’s GeoDjango models.

To get around this problem, we will need to specifically set up a `template_postgis` template database, much like what we did before postgis 2.0 (reference django docs https://docs.djangoproject.com/en/dev/ref/contrib/gis/install/postgis/#creating-a-spatial-database-template-for-earlier-versions).  But because I am running an instance of postgresql 9.1 server and an instance of 9.2 server concurrently, I will need to modify my commands a little.

Like this:-

calvin$ psql92 -p 5433 -U postgres
psql92 (9.2.1)
Type "help" for help.

postgres=# CREATE DATABASE template_postgis ENCODING='utf-8';
CREATE DATABASE
postgres=# UPDATE pg_database SET datistemplate='true' WHERE datname='template_postgis';
UPDATE 1
postgres=# \q

calvin$ POSTGIS_SQL_PATH=/opt/local/share/postgresql92/contrib/postgis-2.0

calvin$ psql92 -p 5433 -U postgres -d template_postgis -f $POSTGIS_SQL_PATH/postgis.sql

calvin$ psql92 -p 5433 -U postgres -d template_postgis -f $POSTGIS_SQL_PATH/spatial_ref_sys.sql

calvin$ psql92 -p 5433 -U postgres -d template_postgis -c "GRANT ALL ON geometry_columns TO PUBLIC;"
GRANT

calvin$ psql92 -p 5433 -U postgres -d template_postgis -c "GRANT ALL ON geography_columns TO PUBLIC;"
GRANT

calvin$ psql92 -p 5433 -U postgres -d template_postgis -c "GRANT ALL ON spatial_ref_sys TO PUBLIC;"
GRANT

With this, running `./manage.py test` with your geodjango-based application will work perfectly fine because django’s default test runner will look for the template_postgis template database to instantiate our now spatially-aware test database.

A Review of Rob Napier, Mugunth Kumar’s iOS 6 book; and advanced debugging with Python + LLDB

I have had the benefit of a sneak preview into Rob and Mugunth’s upcoming book release - iOS 6 Programming Pushing the Limits: Advanced Application Development for Apple iPhone, iPad, and iPad Touch- which is an updated release to their successful iOS 5 pushing the limits series.

As any veteran programmers would know, one of the most important skills (among a multitude of skill requirements) to master is the art of debugging an application.  Chapter 19 provides an excellent coverage of this topic in the context of iOS apps.  Beginning with a conceptual introduction to LLDB (Apple’s Lower Level Debugger) and the difference from the older version of the debugger – GDB – the chapter introduces the intermediate programmer to the actual mechanics of debugging with LLDB and the use of breakpoints (which is no different from the use of breakpoints from many other advanced editors or debugging tools).

Breakpoints

Having mucked around with almost 5 conceptually simple iOS apps, I am no strange to using breakpoints to zero in on problems.  What is interesting, however, is the detailed explanation of different types of breakpoints – exception breakpoints, symbolic breakpoints.  Of particular interest and of learning value to me personally, is the use of Ctrl-Click + “share” on the breakpoint to write a breakpoint state into a `xcshareddata` directory, which can be saved into git and so shared with a fellow developer.

Watchpoints

Watchpoints are also interesting to me, being useful for specifically tracking data mutation events – particularly for global variables that have changed, in the context of Singleton implementations, Core Data persistent store coordinators or API engines.

Advanced Use of Debugger through Python Scripting

For our normal, simple use cases, introspecting objects at breakpoints is a simple matter of typing

po myObject
po myDictionary
po myArray

For scalar variables (e.g. integers or structs), we will of course use

p (int) self.myAge
p (CGPoint) self.view.center

What if we need to search through a very large array containing a large number of objects?  This is where we might have problems manually reading the 10,000 objects that are printed out.  In such a case, we can actually use Python to run a search.  Jumpt into the python shell prompt from the lldb prompt by typing “script”. Then, via the great example from the book:-

>>> import mypython_script
>>> array  = lldb.frame.FindVariable("myArray")
>>> yes_or_no = mypython_script.search_object(array, "<search element>")    # search_object is a method we write in our custom mypython_script .py file.
>>> print yes_or_no

This nice little trick from the book goes straight into my personal bookmark.

Finishing Up

The chapter ties up neatly with an explanation of how NSZombieEnabled flag/environment variable is used to track objects in memory, a summary of various types of Application crashes one commonly encounters and how crash stacks can be collected – “natively” via iTunes as well as on 3rd party services like TestFlight and HockeyApp.

All-in-all, this is a must-have book for any professional iOS developer and certainly a level-up for me.  I have merely covered specifically this one chapter of a very detailed and extensive pro manual and I am certainly looking forward to grabbing a copy (or receiving a copy free ;-)) from Mugunth.  Grab it at Amazon.com if you think this is something that would help you build even more awesome iOS apps - iOS 6 Programming Pushing the Limits: Advanced Application Development for Apple iPhone, iPad, and iPad Touch.