Moved WordPress blog to wordpress.com

I feel about system administration like I feel about changing my car oil. I can do it. But I’d prefer to have a professional do it for me.

I don’t think it’s a bad exercise to run your own WordPress, but I get lazy about doing the upgrades. And I think I get enough practices with devops running my own VPS. So, in the midst of upgrading my VPS, I finally mustered up the resignation to move my WordPress blog to wordpress.com.

An impressively simple operation. Just went into my old WordPress dashboard. Select tools > export and download my old blog’s data as an XML file. Then uploaded that to my new dashboard here. Voila!

Now to find a new theme.

Advertisements
Moved WordPress blog to wordpress.com

Upgraded to WordPress 4.2.2

Someone mentioned to me this week that his company had moved their site off WordPress because it kept getting hacked. Reminded me that I needed to update this site. Jumped from 3.? to the latest.

Wrote a couple Ansible scripts. One to pull down my production database so I could upgrade on my dev machine. The other to put it back up after manually upgrading.

Hit this issue along the way. Manually changing the collation on my local database before running my deploy script proved easier in the end than trying to upgrade MySQL on my server.

I should probably just move to wordpress.com.

Upgraded to WordPress 4.2.2

Functional Tests and Continuous Integration with Google App Engine

I recently decided to get serious about testing my Appswell App Engine project and set up tests to run from the command line with continuous intregration using Jenkins. I’d looked into running tests for App Engine from the command line in the past but inevitably gave it up as being either too brittle or too convoluted. This time I decided not to wuss out.

After a bit of struggle, I finally managed to get a dependable solution wired together using virtualenv and nosetests. There were three key parts to getting the functional tests running in Jenkins:

  • Pip requirements (for virtualenv)
  • Configuration file for nosetests
  • Jenkins build command

Here are the actual scripts for each of the configuration components above. They are provided as a general reference and broad reassurance against that hopeless feeling that others, like me, may have felt when confronting this problem. Of course, some of the details are particular to my environment and you’ll have to adapt them to your own circumstances.

requirements.pip

NoseGAE==0.2.0
WebOb==1.2.3
WebTest==2.0.5
argparse==1.2.1
nose==1.3.0
nose-exclude==0.1.9
six==1.3.0
waitress==0.8.3
wsgiref==0.1.2

nosegae-setup.cfg

[nosetests]
where=./appspot/project/test
tests=model,unit,functional
verbosity=2
exe=1

# nosegae
with-gae=1
without-sandbox=1
gae-application=./appspot
gae-lib-root=./google_appengine_sdk

# requires nose-exclude package
exclude-dir=./appspot/project/test/dev
    ./appspot/project/test/tools

Jenkins virtualenv builder

Note: this uses the Shining Panda plugin: https://wiki.jenkins-ci.org/display/JENKINS/ShiningPanda+Plugin

# vars ref: https://wiki.jenkins-ci.org/display/JENKINS/ShiningPanda+Plugin

# Install pip requirements
pip install -r requirements.pip

# Link Google SDK
rm -f ./google_appengine_sdk
ln -sv /home/klenwell/projects/google/appengine/google_appengine-1.7.6 ./google_appengine_sdk

# Set PYTHONPATH
export PYTHONPATH=./appspot:"$PYTHONPATH"

# Additional tweaks
if [ ! -f ./appspot/config/core.py ]; then
cp ./appspot/config/core.py-dist ./appspot/config/core.py
fi

# Run Tests
$VIRTUAL_ENV/bin/nosetests -c nose.cfg

One final note: I was getting the error mentioned in this nose-gae bug report (AttributeError: ‘module’ object has no attribute ‘BaseRequest’) until I upgraded from Google App Engine SDK 1.7.5 to 1.7.6.

Functional Tests and Continuous Integration with Google App Engine

Tar with Absolute Paths

I recognize the tar interface is not really designed with absolute paths in mind, but I find absolute paths generally easier to work with when scripting.

Indeed, when writing a fabric script that requires sudo on a remote server, you need to use absolute paths because of an issue using sudo with the cd command.

So here’s a simple example of using absolute paths with tar:

# set up dir to tar and zip
mkdir -p ~/tar_me/subdir
touch ~/tar_me/{file1.txt,file2.txt}
touch ~/tar_me/subdir/{file1.txt,file2.txt}

# test input/output directories
mkdir /tmp/tar_in
mkdir /tmp/tar_out

# create zipped tar using absolute paths
tar -vczf /tmp/tar_in/tar_me.tgz -C /home/me/tar_me

# unzip tar using absolute paths
tar -xvf /tmp/tar_in/tar_me.tgz -C /tmp/tar_out/

Before: tree /home/me/tar_me

/home/me/tar_me/
|-- file1.txt
|-- file2.txt
+-- subdir
	|-- file1.txt
	+-- file2.txt

After: tree /tmp/tar_out/tar_me

/tmp/tar_out/tar_me
|-- file1.txt
|-- file2.txt
+-- subdir
	|-- file1.txt
	+-- file2.txt

References

Tar with Absolute Paths

California Population Data

It’s surprisingly hard to find a simple list of annual population data for the state of California. You can find a chart on Google:

http://www.google.com/publicdata/embed?ds=kf7tgg1uo9ude_&ctype=l&strail=false&bcs=d&nselm=h&met_y=population&scale_y=lin&ind_y=false&rdim=state&idim=state:06000&ifdim=state&tstart=333270000000&tend=1248418800000&hl=en&dl=en&uniSize=0.035&iconSize=0.5&icfg

But the underlying data must be extracted from several spreadsheets from the census site.

So here’s a simple Python list of the annual data (1950-2010):

[(1950, 10677000),
 (1951, 11134000),
 (1952, 11635000),
 (1953, 12251000),
 (1954, 12746000),
 (1955, 13133000),
 (1956, 13713000),
 (1957, 14264000),
 (1958, 14880000),
 (1959, 15467000),
 (1960, 15870000),
 (1961, 16497000),
 (1962, 17072000),
 (1963, 17668000),
 (1964, 18151000),
 (1965, 18585000),
 (1966, 18858000),
 (1967, 19176000),
 (1968, 19394000),
 (1969, 19711000),
 (1970, 19971069),
 (1971, 20345939),
 (1972, 20585469),
 (1973, 20868728),
 (1974, 21173865),
 (1975, 21537849),
 (1976, 21935909),
 (1977, 22352396),
 (1978, 22835958),
 (1979, 23256880),
 (1980, 23667902),
 (1981, 24285933),
 (1982, 24820009),
 (1983, 25360026),
 (1984, 25844393),
 (1985, 26441109),
 (1986, 27102237),
 (1987, 27777158),
 (1988, 28464249),
 (1989, 29218164),
 (1990, 29950111),
 (1991, 30414114),
 (1992, 30875920),
 (1993, 31147208),
 (1994, 31317179),
 (1995, 31493525),
 (1996, 31780829),
 (1997, 32217708),
 (1998, 32682794),
 (1999, 33145121),
 (2000, 33987977),
 (2001, 34479458),
 (2002, 34871843),
 (2003, 35253159),
 (2004, 35574576),
 (2005, 35827943),
 (2006, 36021202),
 (2007, 36250311),
 (2008, 36604337),
 (2009, 36961229),
 (2010, 37349363)]

And here’s a list of the raw data and its sources:

# raw data: (year range, raw numbers, unit/multiplier, delim, source)
    raw_data = (
        ( range(1950,1955),
          '10,677   11,134   11,635  12,251   12,746',
          1000,
          's+',
          'http://www.census.gov/popest/archives/1980s/st5060ts.txt' ),
        ( range(1955,1960),
          '13,133  13,713   14,264   14,880  15,467',
          1000,
          's+',
          'http://www.census.gov/popest/archives/1980s/st5060ts.txt' ),
        ( range(1960,1965),
          '15,870   16,497   17,072   17,668   18,151',
          1000,
          's+',
          'http://www.census.gov/popest/archives/1980s/st6070ts.txt' ),
        ( range(1965,1970),
          '18,585   18,858   19,176   19,394   19,711',
          1000,
          's+',
          'http://www.census.gov/popest/archives/1980s/st6070ts.txt' ),
        ( range(1970,1976),
          '19971069  20345939  20585469  20868728  21173865  21537849',
          1,
          's+',
          'http://www.census.gov/popest/archives/1980s/st7080ts.txt' ),
        ( range(1976,1980),
          '21935909  22352396  22835958  23256880',
          1,
          's+',
          'http://www.census.gov/popest/archives/1980s/st7080ts.txt' ),
        ( range(1980,1985),
          '23667902  24285933  24820009  25360026  25844393',
          1,
          's+',
          'http://www.census.gov/popest/archives/1980s/st8090ts.txt' ),
        ( range(1985,1990),
          '26441109  27102237  27777158  28464249  29218164',
          1,
          's+',
          'http://www.census.gov/popest/archives/1980s/st8090ts.txt' ),
        ( range(1999,1989,-1),
          '33145121  32682794  32217708  31780829  31493525  31317179  31147208  30875920  30414114  29950111',
          1,
          's+',
          'http://www.census.gov/popest/archives/1990s/ST-99-07.txt' ),
        ( range(2000,2011),
         '"33,987,977","34,479,458","34,871,843","35,253,159","35,574,576","35,827,943","36,021,202","36,250,311","36,604,337","36,961,229","37,349,363"',
          1,
          ',',
          'http://www.census.gov/popest/intercensal/state/ST-EST00INT-01.csv' ),
    )
California Population Data