Automated deploy from travis-ci via ssh

The other day I wanted to have an automated deployment for a sideproject (callingallpapers.com) of mine. I wanted to be able to merge a PullRequest on github and have Travis-CI deploy the merged files via rsync after all tests succeed. Easy thing I thought, I can’t be the first one attempting that. Looked like I was at least the first one writing about it… Continue reading Automated deploy from travis-ci via ssh

To commit composer.lock or not

Whether to commit a composer.lock-file to a repository or not seems to be a somewhat religious decision. Either you follow Raphael Dohms and commit your lock-file or you follow Marco Pivetta and you don’t.

But I think the truth is somewhat in between.

Recently I decided to create a badge for Badge Poser that shows whether the lock-file is commited or not. It’s something you can see easily from the repo but hey, it’s a fun project!

And instantly I stumbled over a commited lock-file! Because as it seems, commited composer.lock-files can be a blessing or a curse. What happened? I thought – as the project commited their composer.lock-file – it would be a good idea to run a composer install instead of a composer update. After all I should be left with a working installation after the install-command. That’s why we would commit the lock-file in the first place, isn’t it?

But as it turned out, the composer.lock-file was created some time ago with a now hopelessly outdated PHP-version. I have PHP 7 running on my machine, but the composer.lock-file obviously was created using something like PHP 5.3 or PHP 5.4. So instead of getting a running version of the project, I got composer yelling at me that it can’t resolve an installable version of my lock file. Hm.

Version mismatch

So it looks like it’s dependent on the PHP-Version you are running whether the composer.lock can be used or has to be updated. Sadly there’s no hint that tells me which PHP-Version the file has been created with so I will have to test whether an install will work or not.

Entry the “config/platform”-parameter. It allows the creator of a package to specify what PHP-Version should be used to resolve the dependencies regardless of what the actual PHP-Version is. This might have the consequence of installing useless packages as installed packages might not work on the installed PHP-Version.

So one thing to have in mind when committing or receiving a composer.lock-file is, that it contains a set of dependencies, that was working for the committer but it might not work for the user as the environment might have changed to a point where the packages of the composer.lock-file might not be usable anymore. Consider a locked Version of phpunit/phpunit:4.8 because the file was created using PHP 5.5 and you are running PHP 7.

Dev-Mismatch

The second thing that came to my mind was that the lock-file might or might not contain the dev-requirements. So when you are developing a lib you’d want other developers to have the same versions of your dev-tools so it’s a good idea to have them in the lock file. And when you are developing a web-project (website of a certain extend) you might want to use the lock-file to automatically deploy the libs you where testing agains. But as it’s for deployment you’d create the lock-file without the dev-dependencies. That will then leave other developers without the dev-dependencies, so they will have to run a composer update (because composer install will tell you that you should either run composer install --no-dev or composer update) anyway.

And also when you do automated testing against the lowest,locked,latest-strategy you might run into issues with locked packages not being able to run on the current platform even though they are installed without an issue. Have a look for example at https://travis-ci.org/heiglandreas/composerlocktest/jobs/133745846. The test fails due to PHPUnit being installed in the wrong version due to that version being locked in the composer.lock-file

Conclusion

Be careful when you commit your lock-file. Indicate clearly what PHP-Version you used when creating it. Either by providing the suggested version using the platform-Parameter or by pointing it out in the documentation very clear. And remember to only automatically test the locked dependencies with the PHP-Versions you created the composer.lock

Be careful when you use a lock-file. Check whether there is a hint to the provided version in the composer.json file or in the documentation of the project you are using. Have a look at the CI-config-file, it might give you a hint.

Timezones and PostgreSQL

After the aforementioned talk (and in preparation to the next installment of it) I checked how to do the MySQL-Example in PostgreSQL. It’s easily possible, but it takes a bit more time to get around to it. But hey, it’s a tool for grown-ups, isn’t it? 😉

Again we create the table using this snippet:

CREATE TABLE datetime(
  zeit timestamp without time zone,
  zone character varying(255)
)

INSERT INTO datetime (zeit, zone)
VALUES
    ('2014-03-04 12:23:34','Europe/Berlin'),
    ('2016-05-03 23:12:23','Europe/Busingen'),
    ('2016-05-03 23:12:23','America/Chicago')
;

So to select all entries that are before 14:00H in UTC-Timezone in PostgreSQL we’ll use this query:

SELECT * FROM datetime 
WHERE 
    extract(HOUR FROM zeit AT TIME ZONE zone AT TIME ZONE 'UTC')
    <= 14;

Here we tell PostgreSQL to interpret the value in zeit as localtime in timezone zone and then to convert that to timezone UTC. It will output this:

2014-03-04 12:23:34 | Europe/Berlin
2016-05-03 23:12:23 | America/Chicago

Timezones and MySQL

Recently I presented a talk about Timezones (which I hope to present a few more times). During the preparation I stumbled over a a function that eases timezone-based datetime-calculations in MySQL: CONVERT_TZ. You can use it to convert from one timezone into a different one. So let’s say we have a database created like this:

CREATE TABLE `datetime` (
  `id` int(11) unsigned NOT NULL AUTO_INCREMENT,
  `zeit` datetime DEFAULT NULL,
  `zone` varchar(255) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=4 DEFAULT CHARSET=utf8mb4;

INSERT INTO `datetime` (`id`, `zeit`, `zone`)
VALUES
    (1,'2014-03-04 12:23:34','Europe/Berlin'),
    (2,'2016-05-03 23:12:23','Europe/Busingen'),
    (3,'2016-05-03 23:12:23','America/Chicago');

So the dates in the table are in their respective timezones.

To check which Dates are before 14:00 UTC you can use a select-statement like this:

SELECT * FROM datetime 
    WHERE TIME(CONVERT_TZ(zeit, zone, 'UTC')) 
    < "14:00:00";

It will return something like this:

1 | 2014-03-04 12:23:34 | Europe/Berlin
3 | 2016-05-03 23:12:23 | America/Chicago

So with CONVERT_TZ you can convert a datetime (from a datetime-field) from a timezone into a different timezone. And the timezone can also be a field from the table.

The only prerequisit is that the timezone-informations have to be set up in the database. For that you’ll use the servers timezone-informations by calling mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root mysql.

You can find more information on that at https://dev.mysql.com/doc/refman/5.5/en/time-zone-support.html and https://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html#function_convert-tz

Composer and self-signed certificates

Today I wanted to add a package from our internal satis-repository to a composer.json-file. Easy thing!

I added the satis-server as repo to the composer.json like this:

    {
      "repositories": [{
        "type": "composer",
        "url": "https://example.com/satis",
      }]
    }

Fair enough! That’s it! Run composer and be happy:

$ composer require vendor/package

[Composer\Downloader\TransportException]
  The "https:/example.com/satis/packages.json" file could not be downloaded: SSL operation failed with code 1. OpenSSL Error messages:
  error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed
  Failed to enable crypto
  failed to open stream: operation failed


require [--dev] [--prefer-source] [--prefer-dist] [--no-plugins] [--no-progress] [--no-update] [--update-no-dev] [--update-with-dependencies] [--ignore-platform-reqs] [--sort-packages] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--] [<packages>]...

WTF…???

Ah, yes! The servers certificate is signed by our internal Root-CA. As it’s an internal server that’s what we do. But how to get composer to know that? It took me a while. Adding the Rot-Certificate to OpenSSL didn’t bring the expected results (might be due to Homebrew) as didn’t adding the cert to PHPs keystore (might be due to some weird setup on my machine).

But there surely has to be a way of getting satis (or Toran) up and running with self-signed certs!

There were many hacks around that disabled certificate validation altogether but that’s not what I wanted. And after a short question on Twitter I got a few ideas. Alexander Tureks idea was great: Adding “verify-peer” : “false” to the ssl-options of the repository. Sadly that didn’t do the trick. Finally Jordi Boggiano gave me a hint to a feature I hadn’t found in the composer-docs before: Add the RootCA-Certificate to the ssl-options of the repository (and to the project).

So now my composer.json looks like this:

{
  "repositories": [{
    "type": "composer",
    "url": "https://example.com/satis",
    "options" : {
      "ssl" : {
        "cafile" : "myrootca.crt"
      }
    }
  }]
}

The file “myrootca.crt” is a PEM-file that only contains the root-certificate. You can get it by calling openssl x509 -in <(openssl s_client -connect example.com:443 -prexit 2>/dev/null) > myrootca.crt. And myrootca.crt needs to be on the same level as the composer.json in your project.

Thanks Jordi for the fast response!

Hope that helps someone 😉