BTMash

I've been playing around with using the new version of migrate for a little while. But its been on the more boring side and learning how to use migrate with CSV files (which admittedly feels quite good ^_^) And it was only after an email from Tom Camp and in an effort to get my presentation on Migrate ready for NYC camp and Drupal Camp LA. Speaking of presentation, I'd like to thank Andrew/Drewish as I combined a lot of his presentation material from DrupalCon Denver with my material from last year - his slides are excellent.

One of the big changes that went into the 2.4 release of migrate was how fields and their arguments were handled. The difference is that what were previously arguments for Fields (you had to go look at the code for the various field handlers in the Migrate module and figure out how they needed to get formatted) are now available in the migrate ui as field handlers! So you can see exactly what pieces of the field still need to get mapped out (or atleast you have an idea of what you're ignoring). While I have not seen this functionality make its way to contributed field modules (link, entityreference, etc), its a welcome change if you look at the image below:

(Forgive the image width on lower-resolution devices). In the image above, you can now see that what were previously TextFieldHandler argument options are now available as field mappings. They have taken on a tokenized format so that, let's say we have a text field called 'field_course_description', we have potential mappings for 'field_course_description', 'field_course_description:format', 'field_course_description:language', and 'field_course_description:summary' if we were dealing with a text field that included a summary teaser. And this behavior has been extended to pretty much all core fields. Contributed field modules with migrate handlers such as Date/Entityreference still need to catch up to this new change (but arguments will still work just fine for those modules). So you can provide default arguments, an array from a source field...whatever you want and it works beautifully. While there are many more lines of code for a given field mapping (see below), it is much easier to see if the various components have been mapped out and how.

This logic has naturally made its way over to filefield handlers as well (which is the other large change that has come into the 2.4 release of migrate) and instead of writing out something like:

    $file_arguments = MigrateFileFieldHandler::arguments(NULL, 'file_copy', FILE_EXISTS_REPLACE);
    $this->addFieldMapping('field_content_linked_files', 'linked_files')
      ->arguments($file_arguments);

We could now have:

    $this->addFieldMapping('field_content_linked_files', 'redcat_new_migration_linked_files_filename');
    $this->addFieldMapping('field_content_linked_files:file_class')->defaultValue('MigrateFileUri');
    $this->addFieldMapping('field_content_linked_files:file_replace')->defaultValue(FILE_EXISTS_RENAME);
    $this->addFieldMapping('field_content_linked_files:preserve_files')->defaultValue(FALSE);
    $this->addFieldMapping('field_content_linked_files:source_dir')->defaultValue('/path/to/my/files');
    $this->addFieldMapping('field_content_linked_files:description', 'redcat_new_migration_linked_files_description');
    $this->addFieldMapping('field_content_linked_files:display', 'redcat_new_migration_linked_files_display');

Again, more lines of code, but much easier to read. There are a few pieces of caveats in here (namely around the file paths and where files will get saved) that I had to work around. First, the official release will put all the files in one directory if you specify the destination_path (it won't read through the array) - it looks to be fixed in the dev release thanks to this issue. If you want the file to get saved wherever you dictated in your file/image field settings, do not create a mapping for destination_dir or destination_file

Migrating files separately

With the newest release of Migrate and trying out the above, I actually found that another way to approach my scenario (already have site, files were stored in legacy locations, can keep it that way) was to create a file migration. And it is very easy. You create the connection as you need, get the fields containing file information that you need, and go.

    $this->arguments = $arguments;
    parent::__construct();
    $this->description = t('Import All Files');
 
    $source_fields = array(
      'fid' => t('The File ID.'),
    );
 
    $connection = $this->getDatabaseConnection();
    $query = $connection->select('my_files', 'mf');
    $query->fields('mf');
 
    $this->source = new MigrateSourceSQL($query, $source_fields, NULL, array('map_joinable' => FALSE));
    $this->destination = new MigrateDestinationFile('file');
 
    $this->map = new MigrateSQLMap($this->machineName,
      array(
        'fid' => array(
          'type' => 'int',
          'unsigned' => TRUE,
          'not null' => TRUE,
          'alias' => 'fm',
        )
      ),
      MigrateDestinationFile::getKeySchema()
    );
 
    $this->addFieldMapping('uid')->defaultValue(1);
    $this->addFieldMapping('source_dir')->defaultValue($this->getFilesDirectory());
    $this->addFieldMapping('timestamp', 'timestamp');
    $this->addFieldMapping('value', 'uri');
    $this->addFieldMapping('destination_file', 'uri');
    $this->addFieldMapping('file_replace')->defaultValue('FILE_EXISTS_REUSE');
    $this->addFieldMapping('preserve_files')->defaultValue(TRUE);
    $this->addUnmigratedSources(array('uid', 'filename', 'filemime', 'filesize', 'status', 'type'));
  }

And that is it! If you had files in other tables, it might be a little tougher but this is pretty much the gist of it. The advantage of such a scenario is that you can change your destination_file to be whatever you what once you start getting into the prepareRow area for any of your files - it becomes very flexible. As a note, the preserve_files option set above will make sure that the files don't get deleted since I may be doing rollbacks in my migration during the testing phase.

After all that hard work, my file migration in the original page migration looks a little different:

    $this->addFieldMapping('field_content_associated_images', 'redcat_new_migration_associated_images_fid')->sourceMigration('RedcatNewMigrationFiles');
    $this->addFieldMapping('field_content_associated_images:file_class')->defaultValue('MigrateFileFid');
    $this->addFieldMapping('field_content_associated_images:alt', 'redcat_new_migration_associated_images_alt');
    $this->addFieldMapping('field_content_associated_images:title', 'redcat_new_migration_associated_images_title');

Since the files have already been pulled over, just use the file id that was created in the migration process. It maps very quickly. And because files don't have to be transferred over yet again, your content will now migrate over much faster.

I've put up my entire example on bitbucket so you can go through the migration files at your own pace. And as always, I hope all of this has been helpful. If something doesn't make a lot of sense, leave a comment! I want to keep improving this documentation and hopefully, all of this will be even more useful to other folk looking to migrate content in the Drupal community.