Dancers. Photo by Hulki Okan Tabak on Unsplash - (cropped and decolorized)

Hidden Hat Press

Articles

popp crop

POPP Diary

So here’s the thing: my book, PHP Objects, Patterns, and Practice has been commissioned for a sixth(!) edition. As I started my planning this time round, I decided to keep a journal of work done so that I can refer back later on to any discoveries I make on the way. Then it occurred to me that I might leave those notes right here.

So I’m going to try to post a week’s worth of diary entries every Monday between now and my final first draft submission. Of course, most of my editorial effort will be going into the book so the entries may end up pretty rambling. Still, I’ll try to add a tl;dr at the top of each article with a summary to save you trawling through it all if you’re in a hurry or for some reason are not fascinated by the minutiae of tech book revision.

Week summary

This was a discovery and set up week. I raised my game during work on the last edition by writing tests for all my code examples and creating some basic scripts for outputting listings. I seem to remember that the latter did not work without some pretty severe annoyances but I’ll face that when the time comes. The big benefit of having automated testing for listings is that I can keep on checking the code as PHP evolves. This edition of the book relates explicitly to PHP 8 which will not be officially released until well after I have submitted my work. There should be a feature freeze along the way but I need to treat the PHP binary as a moving target as I work – that means I’ll have to go back and test all my code on a regular basis.

Once I found all my old chapters and my code archive, it was time to dust off the vagrant provision script and see what worked. Quite a lot, it turns out. The old provision script needed a little updating but then it worked nicely to get me a virtual machine running Centos 7 with PHP 7.3 and all my tools and goodies. However, I need to be working with the development version of PHP 8 – and I’m not going to get that using the Yum package management system. So it was time to build a script to clone and build PHP from source.

In order to run tests I had to resolve some dependency hell issues (with a terrible hack). Along the way I extended the build script to add various PHP extensions (the first of many). Being able to build everything repeatably from scratch is a key requirement of any project, so I took a bit of time to flesh out the provision scripts before finally updating and fixing my tests.

So, lots of foundational work this week. I’ve learned, though, that investing upfront in tests and scripts for repetitive tasks soon pays off. Unless of course it’s just a way of avoiding doing the real work. I guess knowing the difference is a mark of experience. Though I can’t say I always get it right even now.

Read on for the details – or see you back here next week.

Tuesday 26 May 2020

Found my old source code git archive in bitbucket. I’d forgotten that last edition I finally got everything into vagrant and wrote some imperfect but useful scripts for generating source code listings.

Also found my old chapters and figures locally. So we’re off!

Wednesday 27 May 2020

Updated my vagrant provision script to work with a later PHP than last edition – 7.3 for now. Some other tweaks because I can’t leave well alone. I’m installing into Centos 7 – and my test provision ran fine first time.

I’m quite happy using a shell script for provisioning, but it’s very old school. Maybe I should port my old setup.sh to ansible – and, if I do, I should probably cover that in the new edition.

Cloned https://github.com/php/php-src.git in preparation for daily builds. After years of lazily Yumming hand install might be an adventure.

28 Thursday May 2020

The best starting point to build your own bleeding edge PHP instance is the PHP site – followed by the README.md file (nicely formatted on github).

In order to get to a basic install I first needed to make sure I had some tools and libraries on my Centos 7 VM:

sudo yum install -y autoconf
sudo yum install -y libtool
sudo yum install -y bison
sudo yum install -y re2c

sudo yum install -y libxml2-devel
sudo yum install -y sqlite-devel

Then a script to clone, build and install

if [ ! -d php-src ]; then
    git clone https://github.com/php/php-src.git
fi

cd php-src
git checkout master
git pull origin master
./buildconf

## now from README.md
./configure --enable-debug
make -j4
sudo make install

And, joy of joys!

$ php --version
PHP 8.0.0-dev (cli) (built: May 28 2020 16:53:30) ( NTS DEBUG )
Copyright (c) The PHP Group
Zend Engine v4.0.0-dev, Copyright (c) Zend Technologies

Since PHP is in active development I’m going to be building it a lot. I’ll probably also build my development environment from scratch occasionally. So I’ll add the base requirements to my Vagrant provisioning script and turn the install steps into a more robust script that I can run regularly and extend to handle the various extensions I’m going to be needing.

Friday 29 May 2020

Attempted to install a tool using Composer in Vagrant and hit my first missing extension – OpenSSL. I doubt I’ll document all of these, since it’s going to become extremely repetitive but here’s the process this time. First of all I need the development libraries for OpenSSL

sudo yum install openssl-devel

And then it’s just a matter of adding the argument to the configure call during install

./configure --enable-debug -with-openssl

How did I know to do that? The extension manual page told me about the configure argument. Ensuring the libraries were there was just a matter of knowing the Yum package naming convention.

Onward. What next? Well I should check to see if I can run the tests I wrote last time round using my sparkling new development PHP. Now that I can run composer update again, I can get the tools I need for that. And… my first incompatibility(!):

$ phpunit test/ch03/

Fatal error: Uncaught Error: Call to undefined function each() in /var/popp/vendor/phpunit/phpunit/src/Util/Getopt.php:80
Stack trace:
#0 /var/popp/vendor/phpunit/phpunit/src/TextUI/Command.php(266): PHPUnit_Util_Getopt::getopt(Array, 'd:c:hv', Array)
#1 /var/popp/vendor/phpunit/phpunit/src/TextUI/Command.php(141): PHPUnit_TextUI_Command->handleArguments(Array)
#2 /var/popp/vendor/phpunit/phpunit/src/TextUI/Command.php(132): PHPUnit_TextUI_Command->run(Array, true)
#3 /var/popp/vendor/phpunit/phpunit/phpunit(55): PHPUnit_TextUI_Command::main()
#4 {main}
  thrown in /var/popp/vendor/phpunit/phpunit/src/Util/Getopt.php on line 80

sigh So this is going to be my life for the next few months! It looks like the deprecated each operator which returns the key and value at the current point in an array and advances the pointer has gone away entirely in PHP 8. I’ll need to check that this remains the case.

Looking at the PHPUnit version I installed, I find a extremely venerable PHPUnit 4 which is odd. That’s older than the version I was working with last edition. Turns out, it’s the oldest version that will work with a PHP that is not version 7.n – so Composer gamely installed it for me. Perhaps I should install a bleeding edge PHPUnit? Well that needs the mbstring extension. Off I go, then, down that rabbit hole.

I’m going to have to rebuild PHP. First, I will need add a required library:

sudo yum install oniguruma-devel

And then I can enable the extension before building again

/configure --enable-mbstring --enable-debug -with-openssl

The install goes smoothly and I have a better binary than before, so not a waste of time. BUT no bleeding edge PHPUnit packages will install against PHP version 8.0.0-dev.

So I do a bad thing. I roll back my PHP version to 7.3, install my Composer packages with PHPUnit 9 specified in the composer.json file. That all runs without a hitch. Then, with great sneakiness, I reinstall the development version of PHP. And lo!

$ php --version
PHP 8.0.0-dev (cli) (built: May 28 2020 16:53:30) ( NTS DEBUG )
Copyright (c) The PHP Group
Zend Engine v4.0.0-dev, Copyright (c) Zend Technologies
$
$ phpunit --version
PHPUnit 9.1.5 by Sebastian Bergmann and contributors.

I have the latest PHPUnit – and the old each lines are gone – so I won’t have to hand-hack them!

Of course all my tests are now out of date against this PHPUnit and none of them will run. Update should be pretty straightforward. But that’s another day’s work! It’s Friday so I’m off to start making pizza and put some beer in the fridge.

Saturday 30 May 2020

Well it’s Saturday and the sun is out. I’ve been to the allotment and watered stuff. But how about a check in? I’d quite like to build my vagrant from scratch and see if everything I’ve been doing by hand and then adding to my setup script will run. That’s a bit scary. But here we go.

$ vagrant destroy
    default: Are you sure you want to destroy the 'default' VM? [y/N] y
    ==> default: Forcing shutdown of VM...
    ==> default: Destroying VM and associated drives...
    ==> default: [vagrant-hostmanager:guests] Updating hosts file on active guest virtual machines...
    ==> default: [vagrant-hostmanager:host] Updating hosts file on your workstation (password may be required)...

Although it feels like I’m throwing away all my work here, the whole point of Vagrant is that provisioning should be automated and (more or less) idempotent. I need to be able to rebuild my environment from scratch and provide a version for tech reviewers (though to be fair they often opt to hook up their own systems).

So now it’s time for vagrant up. As I write this, the provisioner is running the compile script. The steps are:

  • some housekeeping around ssh forwarding and Bitbucket/Github acces
  • install many packages via yum - both for development and to support PHP (including devel libraries for extensions)
  • various home directory configuration steps (setting up PATH, vim settings etc)
  • install PHP 7.3 for bootstrapping
  • install some httpd config (not yet checked this time around)
  • install a MariaDB database and tables for use in examples
  • install Composer
  • run composer update in my core directory (/var/popp inside my virtual machine) – this gets me various tools, not least of which is PHPUnit 9.
  • remove PHP 7.3
  • run the new PHP build script that
    • clones the PHP repo (if not done already)
    • pulls the latest master
    • runs configure / make / make install

And did it run successfully the first time? Of course not. I’m currently running my fourth clean provision. Aaand… success! So today I have achieved absolutely nothing. I have arrived back where I started. And I’m well pleased with that result. It means I have a reproducible environment – running the nearest thing to PHP 8 that exists right now.

Sunday 31 May 2020

Back to those tests today. Before I circled back to check my Vagrant provisioning I had arrived at test failures. These were caused by the fact that last time round I was still using an ancient PHPUnit – and now I’m at the cutting edge. I was creating tests that looked like this:

namespace popp\ch03;
 
class Batch01Test extends \PHPUnit_Framework_TestCase
{
	// ...

This is the old PEAR style naming convention – once necessary because of PHP’s lack of namespace support. Now, though, I need to declare my test class like this:

namespace popp\ch03;

use PHPUnit\Framework\TestCase;

class Batch01Test extends TestCase

An easy enough change. Except there are fifteen test class files for Chapter Three alone – a drag to update by hand. A bit of quick and dirty sed to the rescue then:

sed -i s/\\\\PHPUnit_Framework_TestCase/TestCase/ test/ch03/*

to replace the name of the parent class. Then

sed -i '/^namespace/a use PHPUnit\\Framework\\TestCase;' test/ch03/*

to insert the use statement. And now I’m ready to try another test run.

$ phpunit test/ch03/
PHPUnit 9.1.5 by Sebastian Bergmann and contributors.

WW..WW...WR.......                                                18 / 18 (100%)

Time: 00:00.926, Memory: 6.00 MB


Tests: 18, Assertions: 72, Warnings: 5, Risky: 1.

I’ve snipped out the warnings – I’ll return to them during chapter development, but it looks like we’re in business!

You May Also Enjoy