BTMash

(Updated November 10, 2011 - Updated information about file actions. Thank you Patrick Thurmond!)
(Updated June 4, 2011 - File Handler Compatibility with Migrate 2.1)
(Updated June 8, 2011 - Image to show how to write out your destination field names)
Updated August 29, 2012: I wrote out a blog post on the new image/file handler changes from Migrate 2.4+ a few weeks ago which you can read about here. I've not updated this blog post with those changes so take a bit of what you learn from these blog posts, and add in all the things from the new one for anything file related (or just look at the new one since I link out to the new reference codebase from there as well). and hopefully it helps :)

Its been a while since I wrote about using the migrate module to migrate content from various sources. The last time, I covered migrating users into Drupal and this time, I will write out how to migrate content into drupal as nodes. Because the migrate module is so flexible, it makes doing such things quite easy.

The first article discussed the merits of using the Migrate module while the second article was a walkthrough on how to import users.

Much like last time, we first define our migration class

class MyBasicPageMigration extends Migration {
  ...
}

And this class will consist of two functions - one is the class constructor while the other is for massaging/adding extra information on a per row basis.

/**
 * Class Constructor
 */
public function __construct() {
  ...
}
 
/**
 * Add any additional data / clean up data for the current row / node that will be added/updated.
 */
public function prepareRow($current_row) {
  ...
}

Let's start by defining the class constructor.
We need to create the mapping information (for more detail on how the mapping information works, my previous article on migrating users would be a recommended read as I will be diving headfirst into posting the code). Please note that I had already defined some constants before hand: MY_MIGRATION_DATABASE_NAME is the name of the database, MY_MIGRATION_FILES_DIRECTORY is the path to my files directory.

  public function __construct() {
    parent::__construct();
 
    $this->description = t('Migrate basic pages');
 
    $source_fields = array(
      'nid' => t('The node ID of the page'),
      'linked_files' => t('The set of linked files'),
      'right_side_images' => t('The set of images that previously appeared on the side'),
    );
 
    $query = db_select(MY_MIGRATION_DATABASE_NAME .'.node', 'n')
      ->fields('n', array('nid', 'vid', 'type', 'language', 'title', 'uid', 'status', 'created', 'changed', 'comment', 'promote', 'moderate', 'sticky', 'tnid', 'translate'))
      ->condition('n.type', 'page', '=');
    $query->join(MY_MIGRATION_DATABASE_NAME .'.node_revisions', 'nr', 'n.vid = nr.vid');
    $query->addField('nr', 'body');
    $query->addField('nr', 'teaser');
    $query->join(MY_MIGRATION_DATABASE_NAME .'.users', 'u', 'n.uid = u.uid');
    $query->addField('u', 'name');
    $query->orderBy('n.changed');
 
    $this->highwaterField = array(
      'name' => 'changed', // Column to be used as highwater mark
      'alias' => 'n',           // Table alias containing that column
    );
 
    $this->source = new MigrateSourceSQL($query, $source_fields);
    $this->destination = new MigrateDestinationNode('page');
 
    $this->map = new MigrateSQLMap($this->machineName,
      array(
        'nid' => array(
          'type' => 'int',
          'unsigned' => TRUE,
          'not null' => TRUE,
          'description' => 'D6 Unique Node ID',
          'alias' => 'n',
        )
      ),
      MigrateDestinationNode::getKeySchema()
    );
 
   ...
}

From the above code, you should see that it is very similar to the mappings done for the user in the prior link. We are simply defining more tables to join against the initial table and columns that we want pulled in. However, there is one new component that has now been added: highwater. Highwater is a more recent concept that has been added to the migrate module which provides you with the ability to update existing content that had been migrated into your new Drupal site. This means that content does not have to be rolled back and migrated back in! You define which column from your table will be the 'highwater' mark to denote whether or not a piece of content needs to be updated. The other piece that you have to provide is your sql query has to also be sorted by the same highwater column.

    /*
      Nodes will be checked for updates based on the 'changed' column 
      from the original db defined in the sql query above.
    */
    $this->highwaterField = array(
      'name' => 'changed', // Column to be used as highwater mark
      'alias' => 'n',           // Table alias containing that column
    );

With that out of the way, we are now ready to start mapping fields. As a reference, this is what part of my migration screen looks like for migration of various pieces of node info (this includes node title, status, uid, etc along with fields).

Most of the values below are the same as the way in which mappings were done for the users.

    // Make the mappings
    $this->addFieldMapping('title', 'title');
    $this->addFieldMapping('is_new')->defaultValue(TRUE);
    $this->addFieldMapping('uid', 'uid');
    $this->addFieldMapping('revision')->defaultValue(TRUE);
    $this->addFieldMapping('revision_uid', 'uid');
    $this->addFieldMapping('created', 'created');
    $this->addFieldMapping('changed', 'changed');
    $this->addFieldMapping('status', 'status');
    $this->addFieldMapping('promote', 'promote');
    $this->addFieldMapping('sticky', 'sticky');
    $this->addFieldMapping('comment', 'comment');
    $this->addFieldMapping('language')->defaultValue('und');
 
    $this->addFieldMapping('path')->issueGroup(t('DNM'));
    $this->addFieldMapping('pathauto_perform_alias')->defaultValue('1');
 
    $this->addFieldMapping(NULL, 'name');
    $this->addFieldMapping(NULL, 'vid');
    $this->addFieldMapping(NULL, 'type');
    $this->addFieldMapping(NULL, 'language');
    $this->addFieldMapping(NULL, 'moderate');
    $this->addFieldMapping(NULL, 'tnid');
    $this->addFieldMapping(NULL, 'translate');

But the migration wouldn't be a migration if we only have simple node table column values to migrate and had actual fields. The migrate module has currently mapped out all core fields (and migrate extras is mapping out fields defined by other modules though work is ongoing). In the case of text fields (or text field with summary/whatnot), there are now arguments to migrate that content into your new Drupal site.

    $body_arguments = MigrateTextFieldHandler::arguments(NULL, filter_default_format(), NULL);
    $this->addFieldMapping('field_content_body', 'body')
      ->arguments($body_arguments);

The first argument in the body_arguments is to see which field would map to a summary text field (if there was any). In the event that we did have one, it would be by defining where to find the source field:
$arguments = MigrateTextFieldHandler::arguments(array('source_field' => 'excerpt'));. The second field is which filter you wish to apply to the content (so if you know the machine name of the filter you wish to use or pull it dynamically, you may do so in the same way as for the first argument). The final argument is the language of the text field.

The migrate module also provides the ability to import file fields and have them get attached to nodes.

    $associated_file_arguments = MigrateFileFieldHandler::arguments(NULL, 'file_copy', FILE_EXISTS_RENAME);
 
    $this->addFieldMapping('field_content_associated_images', 'right_side_images')
      ->arguments($associated_file_arguments);

Update as of June 4, 2011: There is a lot less going on in the code above - we get going to be getting multiple files and assigning attributes to them for the import. For the arguments, they consist of:

1. Source path to the file.
2. File action: file_move, file_copy, file_fid, file_link, file_blob, or file_fast (which doesn't really do anything)
3. Action if file already exists.

If the first item is null, then defining it in your values passed in will need to define a path to the file (this will all get defined during the prepareRow() implementation. Also for this file definition, things are *MUCH* simpler than using the separator method from Migrate 2.0 whereby a PHP array is defined (and JSONified using drupal_json_encode) for migrate. For the array, we define a few things:

1. Source Path to the file
2. Language
3. File Alt Text
4. File Title Text
5. File Description
6. File Display name

  public function prepareRow($current_row) {
    // Right Side Images
    $right_side_image_query = db_select(MY_MIGRATION_DATABASE_NAME .'.content_field_right_side_image', 'rsi')
      ->fields('rsi')
      ->condition('rsi.nid', $nid, '=');
    $right_side_image_query->join(MY_MIGRATION_DATABASE_NAME .'.files', 'f', 'rsi.field_right_side_image_fid = f.fid');
    $right_side_image_query->addField('f', 'filepath');
    $results = $right_side_image_query->execute();
 
    $images = array();
 
  foreach ($results as $row) {
    $image_data = unserialize($row->field_right_side_image_data);
    $current_image_data = array(
      'path' => REDCAT_MIGRATION_FILES_DIRECTORY .'/'. $row->filepath,
      'alt' => $image_data['alt'],
      'title' => $image_data['title'],
    );
    $images[] = drupal_json_encode($current_image_data);
  }
    $current_row->right_side_images = $images
    return TRUE;
  }

Since the right_side_images above are now an array of images, you can have 0, 1, 10, 20, N files associated with that particular field! And with all that, we have our various fields defined for migrating content from another database into nodes!
I hope all of this is helpful. If something doesn't make a lot of sense, leave a comment! I'll try and improve this documentation. And next time, I'll post how to migrate an event node with dates, taxonomy terms, and attached files.