Symfony on Google App Engine

The Symfony on Google App Engine Standard Tutorial provides a quick-start on deploying Symfony to Google App Engine’s Standard PHP environment but it is missing a couple details and doesn’t demonstrate some features many apps would need like SQL.  I’ve been working on deploying a more complicated example project to GAE-Standard so I thought I’d note the issues I’ve run into.

Platform

My app is using Symfony 3.4, Doctrine ORM 2, and FOS Rest 2.  It relies on a MySQL database. The code is hosted on my local GitLab server. My development machine is a Fedora Linux laptop.

I’m targeting GAE’s Standard PHP Environment rather than their Flexible PHP  Environment so I can save some costs.  The Standard Environment has a free tier and can scale down to zero when not in use making development costs most likely zero. For that cost savings, we have to deal with some limitations. The read-only file system and a limit on the number of files that can be deployed are the biggest issues I found.

I setup a project in the Cloud Console to get started.  I created a stock Symfony 3.4 project as well.

Database

The database I’m using is Google’s Cloud SQL for MySQL. This provides a minimal VM running an instance of MySQL.  I chose their “gen-2” version then created a database and user for the project using the Cloud Console. I then downloaded and installed the proxy tool so I can access the database from my development machine. I run it with the -dir=/cloudsql option so it created a local UNIX-domain socket in that directory since this is how the production environment is automatically setup.

Add Doctrine to the project with composer require doctrine/doctrine-bundle. I had to add an extra database_socket parameter to parameters.yml (and .dist) and adjusted the Doctrine configs accordingly. Note that database_host needs to be set to localhost, not 127.0.0.1, for the driver to use the socket instead of TCP. I also added a couple other settings – charset and server_version. That last one prevents Doctrine from trying to connect to the database early in the deployment when the database isn’t actually there yet.

doctrine:
    dbal:
        driver: pdo_mysql
        host: '%database_host%'
        port: '%database_port%'
        dbname: '%database_name%'
        user: '%database_user%'
        password: '%database_password%'
        unix_socket: '%database_socket%' # added
        charset: utf8mb4                 # added
        server_version: 5.7              # added

 

Sessions

The GAE-Standard filesystem is read-only and any number of instances maybe running in different VMs so we can’t use the default session caching logic. Memcache is automatically provided so we use it. We add a couple entries to services.yml as shown below. Note that the starter-project uses Memcache (without the “d”) but that’s deprecated so we’ve adjusted it a bit here.

 memcached:
     class: Memcached
 session.handler.memcached:
     class: Symfony\Component\HttpFoundation\Session\Storage\Handler\MemcachedSessionHandler
     arguments:
         - "@memcached"

Then in config.yml, we have this:

framework:
    session:
        handler_id: session.handler.memcached
        save_path: ~

Since we’re using Memcached for sessions, we let Doctrine use it as well with this:

doctrine:
    orm:
        metadata_cache_driver: memcached
        result_cache_driver: memcached
        query_cache_driver: memcached

I don’t really understand this last bit.  The starter-project adds a couple more services and a custom MemcachedAdapter class to address what it explains are differences between the Memcache Proxy provided by GAE and an actual instance of Memcache.  Per an issue created in the starter project, we’re using these additional services and it appears to be working.

cache.default_memcached_provider:
    class: AppBundle\Cache\Adapter\MemcachedAdapter
    factory: ['AppBundle\Cache\Adapter\MemcachedAdapter', createConnection]
    arguments:
        - "memcached://localhost"
cache.system: "@cache.adapter.appbundle_memcached"
cache.adapter.appbundle_memcached:
    class: AppBundle\Cache\Adapter\MemcachedAdapter
    abstract: true
    public: true
    tags:
        - name: cache.pool
    provider: cache.default_memcached_provider
    clearer: cache.default_clearer
        - name: monolog.logger
    channel: cache
    arguments:
        - ~ # Memcached connection service
        - ~ # namespace
        - 0 # default lifetime
    call:
        - method: setLogger
        arguments: ['@logger']

We have a copy of the MemcachedAdapter class from the starter-project in our project to support this.

Twig

The read-only file system mean we need to warmup the cache for the prod environment on our development machine then deploy the results up to GAE.  It seems that some of the caching the Twig does differs between versions of PHP so the starter-project has a modified version of the stock Twig\Environment class to deal with this.  We made a copy of the class in our project and hooked up with another addition to services.yml.

twig:
    class: AppBundle\Twig\Environment
    arguments:
        - "@twig.loader"
    call:
        - method: addGlobal
          arguments: [app, "@twig.app_variable"]
        - method: addRuntimeLoader
          arguments: ["@twig.runtime_loader"]
    configurator: ["@twig.configurator.environment", configure]

Deploy

We created an app.yaml file for the project along the same lines as the starter-project.  We made a number of additions to the skip_files section to  reduce the number of files that get deployed. My bin/deploy script looks like so:

#!/bin/bash
project=...
version=...
bin/console cache:clear --no-warmup --env=prod
bin/console cache:warmup --env=prod
gcloud app deploy -q --project $project -v $version

Once that’s run, I can get to the app at http://${version}-dot-${project}.appspot.com/.  Since I’ve not created any new routes, I get the standard “Welcome to Symfony” page.  I can browse the non-static files that were deployed under Debug and see logs under Logging in the Cloud Console.

Environments

I ended up fiddling with the stock Symfony configs so I didn’t need to change the parameters files all the time.  I have separate app/config/parameters_*.yml files for dev, test, and prod as well as a custom init environment I added. I have separate config_*.yml files as well.  This way, the changes above to add services in prod aren’t actually added in dev. The test settings are used when I run phpunit either locally on my development machine or in my GitLab CI environment.

The extra init environment is used when I need to initialize the production database scheme.  My bin/initdb script fires up the Cloud SQL Proxy then runs the doctrine:schema commands after asking “Are you *really* sure?” a couple times.

Problems

I’m running into a few problems as I add functionality to the project.

Twig Issues

There’s something still missing from the modified Twig\Environment object. When the csrf_token() function is used in a template, I’m getting an error from the baseclass’ getRuntime() method indicating it was unable to load Symfony\Bridge\Twig\Form\TwigRenderer. I worked around it by passing the token in from the controller instead of using the function in the template.

Then I ran into another one – app.user is always empty.  I use this to decide in a template whether to emit a /login or /logout link in a header. Doesn’t work but if I pass in the results of $this->getUser() from the controller then use that instead, it works.

When I tried adding EasyAdminBundle, loading the admin page produces another apparent Twig error – Could not parse property path "[entities][]". Unexpected token "[" at position 10.  That appears to be from javiereguiluz/easyadmin-bundle/src/Resources/views/default/list.html.twig but I’m not certain.

Multiple is_writable() calls

The FileCache objects from JMS Serializer and Doctrine (used by EasyAdmin) throw exceptions if their cache directories aren’t writable. The code isn’t actually trying to write there at runtime because we’ve warmed up the cache before deploying but I had to patch those two constructors commenting out the is_writeable() tests to make things work.

I added a couple entries under symfony_scripts in composer.json so the patches are applied automatically.

 "scripts": {
     "symfony-scripts": [
         "patch -p0 -N -r - <asset/jms-FileCache.patch || true",
         "patch -p0 -N -r - <asset/doctrine-FileCache.patch || true",
         "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::buildBootstrap",
         "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::clearCache",
         "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installAssets",
         ...

Still working….

Will update this as I progress.