Migrate Confluence XML export to MediaWiki import data

This is a command line tool to convert the contents of a Confluence space into a MediaWiki import data format. See also the official BlueSpice Helpdesk entry.

Docker

The migrate confluence tool is available as docker image.

Workflow

Export "space" from Confluence

1. Create an export of your confluence space

Step 1:

Step 2:

Step 3:

2. Save it to a location that is accessbile by this tool (e.g. /tmp/confluence/input/Confluence-export.zip)
3. Extract the ZIP file (e.g. /tmp/confluence/input/Confluence-export)
   1. The folder should contain the files entities.xml and exportDescriptor.properties, as well as the folder attachments

Migrate the contents

1. Create the "workspace" directory (e.g. /tmp/confluence/workspace/ )
2. From the parent directory (e.g. /tmp/ ), run the migration commands
   1. Run docker run -v $(pwd)/confluence:/data bluespice/migrate-confluence:latest analyze --src=/data/input --dest=/data/workspace to create "working files". After the script has run you can check those files and maybe apply changes if required (e.g. when applying structural changes).
   2. Run docker run -v $(pwd)/confluence:/data bluespice/migrate-confluence:latest extract --src=/data/input --dest=/data/workspace to extract all contents, like wikipage contents, attachments and images into the workspace
   3. Run docker run -v $(pwd)/confluence:/data bluespice/migrate-confluence:latest convert --src=/data/workspace --dest=/data/workspace (yes, --src /data/workspace/ ) to convert the wikipage contents from Confluence Storage XML to MediaWiki WikiText. For large spaces, see Parallel convert below.
   4. Run docker run -v $(pwd)/confluence:/data bluespice/migrate-confluence:latest compose --src=/data/workspace --dest=/data/workspace (yes, --src /data/workspace/ ) to create importable data

If you re-run the scripts you will need to clean up the "workspace" directory!

Import into MediaWiki

1. Copy the diretory "workspace/result" directory (e.g. /tmp/confluence/workspace/result/) to your target wiki server (e.g. /tmp/result)
2. Go to your MediaWiki installation directory
3. Make sure you have the target namespaces set up properly. See workspace/space-id-to-prefix-map.php for reference.
4. Make sure $wgFileExtensions is setup properly. See workspace/attachment-file-extensions.php for reference.
5. Use php maintenance/importImages.php /tmp/result/images/ to first import all attachment files and images
6. Use php maintenance/importDump.php /tmp/result/pages.xml to import the actual pages

You may need to update your MediaWiki search index afterwards.

Config file

It is possible to use a yaml file to configure the commands analyze, extract and convert. As an example see /doc/config.sample.yaml. The configuration file can be applied by adding the option --config /data/config.yaml.

Not all parameters of config.sample.yaml have to be used in the config file. If something is not part of it the default will be used.

Parallel convert

For large Confluence spaces the convert step can be slow. You can speed it up by running multiple worker processes in parallel using the --workers option.

docker run -v $(pwd)/confluence:/data bluespice/migrate-confluence:latest convert \
  --src=/data/workspace --dest=/data/workspace \
  --workers=4

The command spawns the requested number of child processes automatically. Each worker handles a disjoint slice of the file list, so every file is converted exactly once. Progress lines are prefixed with [Worker N] so you can follow each process individually. If any worker fails the command exits with a non-zero status and reports which workers were affected.

Choose --workers based on the number of available CPU cores. A value between 2 and 8 is typical; there is no benefit in exceeding the number of cores on your machine.

Note: --workers=1 (the default) behaves identically to running without the option — no child processes are spawned.

Extension:NSFileRepo compatibility

There is now a compatibility for the mediawiki extension https://www.mediawiki.org/wiki/Extension:NSFileRepo which restricts access files and images to a given set of user groups associated with protected namespaces.

If NSFileRepo is used the upload of the images can not be done with the script maintenance/importImages.php but with extensions/NSFileRepo/maintenance/importFiles.php.

Example: php extensions/NSFileRepo/maintenance/importFiles.php /tmp/result/images/

User spaces

In confluence user spaces are protected. In MediaWiki this is not possible for namespace User. Therefore user spaces are migrated to a namespace User<username> which can be protected in BlueSpice for MediaWiki.

Included MediaWiki wikitext templates

• AttachmentsSectionEnd
• AttachmentsSectionStart
• Details
• DetailsSummary
• Excerpt
• ExcerptInclude
• Info
• InlineComment
• Layout
• Layouts.css
• Note
• Panel
• RecentlyUpdated
• SubpageList
• SubpageListRow
• Tip
• Warning
• PageTree
• SpaceDetails
• ViewFile

Be aware that those pages may be overwritten by the import if they already exist in the target wiki.

Included upload files

• Icon-info.svg
• Icon-note.svg
• Icon-tip.svg
• Icon-warning.svg

Be aware that those files may be overwritten by the import if they already exist in the target wiki.

MediaWiki settings

In case your pages contain a lot of external images (<img /> elements), be aware that MediaWiki does not show them by default. You'd need to configure $wgAllowExternalImages. Read https://www.mediawiki.org/wiki/Manual:$wgAllowExternalImages for more information.

Jira interwiki links

Confluence pages that contain Jira macros are converted to use MediaWiki interwiki links. Two separate prefixes are used because Jira issue keys and JQL queries have different URL patterns:

Interwiki prefix	Purpose	Example URL pattern
jira	Link to a specific Jira issue by key	https://jira.example.com/browse/$1
jira-jql	Link to a Jira issue list filtered by JQL	https://jira.example.com/issues/?jql=$1

Add both entries to the interwiki table of your MediaWiki database, or configure them via $wgExtraInterlanguageLinkPrefixes and the interwiki cache. Replace https://jira.example.com with the base URL of your Jira instance.

Required MediaWiki extensions

The output generated by the tool contains certain elements that need additonal extensions to be enabled.

1. TemplateStyles
2. ParserFunctions
3. DateTimeTools
4. Checklists
5. SimpleTasks
6. EnhancedUploads
7. Semantic MediaWiki
8. HeaderTabs
9. SubPageList
10. TableTools

Recommended MediaWiki extensions

These extensions are not strictly required but are recommended for full compatibility with the migrated content.

1. WikiMarkdown - Renders <markdown> tags produced from Confluence markdown macros

Manual post-import maintenance

Cleanup Categories

In the case that the tool can not migrate content or functionality it will create a category, so you can manually fix issues after the import

• Broken_link
• Broken_user_link
• Broken_page_link
• Broken_image
• Broken_layout
• Broken_macro/<macro-name>

Not migrated

• User identities
• Comments
• Various macros
• Various layouts
• Blog posts
• Files of a space which can not be assigned to a page

Creating a build

1. Clone this repo
2. Run composer update --no-dev
3. Run box compile to actually create the PHAR file in dist/. See also https://github.com/humbug/box

TODO

• Reduce multiple linebreaks (<br />) to one
• Remove line breaks and arbitrary fromatting (e.g. <b>) from headings
• Mask external images (<img />)
• Preserve filename of "Broken_attachment"
• Merge multiple <code> lines into <pre>
• Remove bold/italic formatting from wikitext headings (e.g. === '''Some heading''' ===)
• Fix unconverted HTML lists in wikitext (e.g. <ul><li>==== Lorem ipsum ====</li><li>'''<span class="confluence-link"> </span>[[Media:Some_file.pdf]]'''</li></ul><ul>)
• Remove empty confluence storage format fragments (e.g. <span class="confluence-link"> </span>, <span class="no-children icon">)