User:Daveh/Wiki Upgrade 1.25

Notes on the MediaWiki upgrade from 1.24 to 1.25.

Upgrade Notes

• The OpenSearchXml extension was merged into MW core.
• Database Upgrade Time (on dev from 1.24) = 2 minutes
• The deprecated entrypoint of SpecialRenameuser.php is being used. It will be removed in a future release. Use Renameuser.php instead.
• The Graph extension works but still uses the old version 1 Vega data format which is why none of the newer examples on the extension page work (all use version 2).

Upgrade Procedure

• Create wiki db copy in the uesp_net_wikidev database on backup1.
• Create copy of wiki images (preferably copy from backup instead of files1 for lag/load reasons.
• Download and extract 1.25.6 from https://releases.wikimedia.org/mediawiki/
• Copy to new directory: copy -Rp ./ /home/uesp/dev/www/w125
• Remove images/ from new wiki and create symbolic link to temporary image directory.
• Copy from existing wiki:

• LocalSettings.php
• config/
• skins/UespMonoBook/
• skins/uespvector/
• Extensions ()

• Grab with: diff -q ./ /home/uesp/dev/www/w125/extensions/ | grep "Only in ./" | awk '{print $4}'
• Copy with: diff -q ./ /home/uesp/dev/www/w125/extensions/ | grep "Only in ./" | awk '{print $4}' | xargs -I{} cp -Rp {} /home/uesp/dev/www/w125/extensions/
• AbuseFilter
• AntiSpoof
• CharInsert
• CheckUser
• CirrusSearch
• DaedricFont
• DailyEdits
• DeleteBatch
• Disambiguator
• DismissableSiteNotice
• DragonFont
• DynamicFunctions
• Editcount
• Elastica
• EsoCharData
• FalmerFont
• Graph
• JobQueue
• JsonConfig
• LabeledSectionTransclusion
• LogPageRenderTimes
• MediaFunctions
• MetaTemplate
• MobileFrontend
• MwEmbedSupport
• Patroller
• ProtectSection
• qwebirChat
• RecentPopularPages
• RegexFunctions
• SearchLog
• StringFunctions
• TimedMediaHandler
• TorBlock
• UespCustomCode
• UespEsoItemLink
• UespEsoSkills
• UespLegendsCards
• UespMap
• UncategorizedUESPPages (not currently used due to bug)
• UsersEditCount
• WikiTextLoggedInOut

• Move new w125 directory to w (move existing as needed).
• Update database with: php ./maintenance/update.php --uespdev
• Upgrade extensions (not including default and UESP custom extensions):

• Command: uesp-getmwext NAME 1_25
• Change require_once(...) to wfLoadExtension(...)
• DynamicFunctions -- No longer maintained.
• LabeledSectionTransclusion -- Remove include LabeledSectionTransclusion/lsth.php
• MwEmbedSupport -- No GitHub, included in MW +1.32
• Patroller -- Only source for 1.26 or later?
• ProtectSection -- No updates
• StringFunctions -- No updates, included optionally in ParserFunctions

• ElasticSearch

• Delete previous indexes with: curl -XDELETE files1.uesp.net:9200/uesp_net_wikidev_current
• Recreate Dev Wiki index: php ./extensions/CirrusSearch/maintenance/updateSearchIndexConfig.php --uespdev --startOver
• You should now be able to run searches without an error message but there will be no results.
• Run: php ./extensions/CirrusSearch/maintenance/forceSearchIndex.php --uespdev --queue --maxJobs 10000 --pauseForJobs 1000 --skipLinks --indexOnSkip --buildChunks 250000
• The last command will output a bunch of lines like: php ./extensions/CirrusSearch/maintenance/forceSearchIndex.php --uespdev --queue --maxJobs 10000 --pauseForJobs 1000 --skipLinks --indexOnSkip --fromId 4 --toId 25000
• Run manually one by one or in parallel. Jobs are just queued so this step is relatively fast (50 pages/sec).
• Start the dev job queue runner: uesp-devmwjobrunner
• IDs 25k-50k takes 7 min to queue (19051 pages at 51/sec) and 52 min to run the 1419 jobs.
• Can start several job queue runners for more speed (x3 results in loads up to 4 on files1 for this step).
• Total time for this step is 4-6 hours using 3 job runners.
• Run: php extensions/CirrusSearch/maintenance/forceSearchIndex.php --uespdev --queue --maxJobs 10000 --pauseForJobs 1000 --skipParse --buildChunks 25000
• Last step outputs a bunch of commands like: php ./extensions/CirrusSearch/maintenance/forceSearchIndex.php --uespdev --queue --maxJobs 10000 --pauseForJobs 1000 --skipParse --fromId 4 --toId 25004
• Average queue insert rate for one script is 50/second.
• Run these scripts using the job runners as before. This step puts more load on files1 so you may want only one job runner (load ~11-14 with one job runner...probably less if files1 wasn't resyncing at the time).
• IDs 4-25k take ~10 minutes to run (files1 under resync).
• IDs 25-50k queue at 116/sec and takes 7 minutes to run (with 1 job runner files1 load increase not noticeable).
• IDs 50-283k queue at 142/sec and takes ~60 minutes to run (with 3 job runners files1 load increases to 10-13, with 2 job runners files1 load increases to 7-13, with 1 job runner files1 load increases to 4-6).

• Extensions

• Change to wfLoadExtension() format for any extension that includes the extension.json file.
• Add CiteThisPage extension.
• For UESP custom extensions move wgExtensionCredits[] to extension.json (see https://www.mediawiki.org/wiki/Manual:Developing_extensions#Setup).

• This seems to be problematic as everything in the main Extension.php has to be moved to the JSON file. require_once() method still seems to be working fine.

• Prefix searching not working on main wiki. Edit resources/src/mediawiki/mediawiki.searchSuggest.js line 130 to be:

   namespace: '0|102|104|106|108|110|112|114|116|118|120|122|124|126|128|130|132|134|136|138|140|142|144|146|148|150|152',

• Prefix searching not working on mobile wiki. Edit extensions/MobileFrontEnd/javascripts/modules/search/SearchApi.js line 54to be:

   searchNamespace: '0|102|104|106|108|110|112|114|116|118|120|122|124|126|128|130|132|134|136|138|140|142|144|146|148|150|152',

PHP 5.6 Testing

• Upgrade test on content3 in preparation for MediaWiki versions 1.27-1.30.
• Follow steps here for getting PHP 5.6 installed from CentOS 6 repositories.
• Install php56 and all require modules:

    yum install php56
    yum install php56-XXX

• Php53 modules installed on content3:

    php
    php-bcmath
    php-cli
    php-common
    php-devel
    php-gd
    php-mbstring
    php-mcrypt
    php-mysql
    php-pdo
    php-pear.noarch
    php-pecl-apc
    php-pecl-memcache
    php-snmp
    php-xml

• Php56 modules installed on content3:

    php56
    php56-php
    php56-php-bcmath
    php56-php-cli
    php56-php-common
    php56-php-devel
    php56-php-fpm
    php56-php-gd 
    php56-php-mbstring
    php56-php-mcrypt
    php56-php-mysqlnd 
    php56-php-pdo
    php56-php-pear.noarch
    php56-php-pecl-apcu 
    php56-php-pecl-jsonc
    php56-php-pecl-jsonc-devel 
    php56-php-pecl-lua
    php56-php-pecl-memcache
    php56-php-pecl-zip
    php56-php-process 
    php56-php-snmp 
    php56-php-xml
    php56-runtime

• Will be installed as php56 and not overwrite the existing php installation.
• Remove /etc/httpd/conf.d/php.conf and check the new php56-php.conf.
• Edit config file /opt/remi/php56/root/etc/php.ini as needed.

    max_execution_time = 300
    memory_limit = 4512M
    post_max_size = 60M
    upload_max_filesize = 100M
    allow_url_fopen = On
    mysql.connect_timeout = 300
    mysql.wait_timeout = 300
    mysqli.reconnect = On

• Restart Apache and check server-status/phpinfo to ensure it is running the version 5.6.
• To change the PHP version used in the shell:

    mv /usr/bin/php /usr/bin/php53
    ln -s /usr/bin/php56 /usr/bin/php

Issues

• Mobile site has white background.

• Add the ext.UespCustomCode.mobile.styles module which adds the following CSS:

    .animations #mw-mf-page-center {
         background-color: none;
    }

• The deprecated entrypoint of SpecialRenameuser.php is being used. It will be removed in a future release. Use Renameuser.php instead

• Being used in the Renameuser extension.
• Change SpecialRenameuser.php to Renameuser.php in Extensions.php.

• Remove the DynamicFunctions extension:

• Move implementation of #arg, #skin, and #rand to MetaTemplate.

Extensions

• Included in MW 1.25.3

• Cite
• CiteThisPage
• ConfirmEdit
• Gadgets
• ImageMap
• InputBox
• Interwiki
• LocalisationUpdate
• Nuke
• OpenSearchXml (included in core)
• ParserFunctions
• PdfHandler
• Poem
• Renameuser
• SpamBlacklist
• SyntaxHighlight_GeSHi
• TitleBlacklist
• WikiEditor

• Extensions Not Being Used

• LogPageRenderTimes -- Performance reasons.
• SearchLog -- Performance reason and probably needs upgrading.
• StringFunctions -- Included with parser functions?
• UncategorizedUESPPages -- Has bug that crashes page. No longer used and seems to provide same feature as the existing "Uncategorized Pages".

• Extensions to Remove

• DynamicFunctions -- Necessary functions moved to MetaTemplate.
• StringFunctions -- No longer needed (merged into ParserFunctions).
• qwebirc -- No longer used?
• UncategorizedUESPPages -- No longer used or needed.

• Added Extensions

• CiteThisPage
• DisableAccount

• UESP Custom Extensions

• DaedricFont
• DailyEdits
• DragonFont
• EsoCharData
• FalmerFont
• JobQueue
• LogPageRenderTimes
• MetaTemplate
• RecentPopularPages
• UespCustomCode
• UespEsoItemLink
• UespEsoSkills
• UespLegendsCards
• UespMap
• UncategorizedUESPPages
• UsersEditCount

References

• https://www.mediawiki.org/wiki/Compatibility
• https://www.tecmint.com/install-php-5-4-php-5-5-or-php-5-6-on-centos-6/
• https://docs.nextcloud.com/server/13/admin_manual/installation/php_56_installation.html