Cypress-Dogwood升级方案

https://groups.google.com/forum/#!searchin/edx-code/migrate|sort:date/edx-code/FZdfXAaawsk/m-Tywq3hGQAJ


Hi all,

I have a cypress release running in my setup. Currently, I am planning to upgrade to dogwood. But I am not sure about the approach we need to follow to achieve the same.

I have following approaches in my mind,

1.) Running the cypress to dogwood migration script on the production platform.
- Risky since it’s my production platform and I am not sure about the downtime.
- In the Worst case, something can go wrong which might end up in data loss.

2.) Setup a new dogwood server. Backup mysql & mongo data from cypress and restore it in the new server
- I tried it using following commands
mongodump -o ./mongo-backup
mongorestore -drop ./mongo-backup
mysqldump -u root -p --all-databases > ./backup.sql
mysql -u root -p < ./backup.sql
- But in this approach i got lot of errors while importing mysql backup in dogwood since the table structure and foreign key reference has been changed.

Right now, I am totally confused and I don’t know which approach should I follow to achieve this.

Can someone help me in this regard and also I would like to know the best practices we need to follow. Looking Forward.

Thanks.


The way we successfully migrated one of our cypress deployments to dogwood while trying to minimize downtime was using these steps:

  1. Prepare THREE new servers. If you’re using ansible, make sure to set the migrate_db variable to false (you will want to perform the migrations manually rather than have ansible do it at deploy time):
  • checkout the tag “release-2015-11-09” on the first one (this is the last release before django 1.8 upgrade)
  • checkout the tag “dogwood-first-18” on the second new server (this is the first django 1.8 release)
  • checkout “named-release/dogwood” on the third new server (this is the final dogwood release)
  1. Once all three servers are ready, backup your mysql/mongo databases.

  2. SSH into the first new server (the one hosting “release-2015-11-09” code), and migrate edx-platform and xqueue databases:
    sudo -Hu edxapp bash
    cd
    source edxapp_env
    cd edx-platform
    ./manage.py lms syncdb --migrate --settings aws && ./manage.py cms syncdb --migrate --settings aws

exit

sudo -Hu xqueue bash
cd
source venvs/xqueue/bin/activate
cd xqueue
SERVICE_VARIANT=xqueue /edx/app/xqueue/venvs/xqueue/bin/django-admin.py syncdb --migrate --settings=xqueue.aws_settings --pythonpath=/edx/app/xqueue/xqueue

  1. SSH into the second new server (the one hosting “dogwood-first-18” code), and migrate edx-platform and xqueue databases, using --fake-initial flag:

sudo -Hu edxapp bash
cd
source edxapp_env
cd edx-platform
./manage.py lms migrate --settings=aws --fake-initial --noinput && ./manage.py cms migrate --settings=aws --fake-initial --noinput

exit

sudo -u xqueue SERVICE_VARIANT=xqueue /edx/app/xqueue/venvs/xqueue/bin/python /edx/app/xqueue/xqueue/manage.py migrate --settings=xqueue.aws_settings --noinput --fake-initial

  1. Re-run ansible scripts on the third server (the one hosting final “named-release/dogwood” code), but this time set migrate_db to true. Alternatively, you could run the migration scripts manually, but we found it was easier to just re-run ansible.

  2. You are done! The third server is now running dogwood and the database has been successfully migrated. You can decommission the other two servers (“release-2015-11-09” and “dogwood-first-18”).

I strongly suggest you try the above procedure using a copy of your production database before doing it on the actual production database.

Matjaz
@OpenCraft


You could use a single server for the upgrade, but you would still have to follow the same steps - upgrade to latest pre-django1.8 release first, run migrations, update to first django1.8 release, run migrations (with --fake-initial) flag, then upgrade to final dogwood release and run migrations again. The benefit of using three separate servers is minimizing downtime - you can prepare all three server in advance, and then quickly run the migrations on each server, without having to do time consuming version upgrades between each step.

The reason upgrading is so complicated is that two large changes happened between cypress and dogwood releases: python was upgraded from 2.7.3 to 2.7.10 and django was upgraded from 1.4 to 1.8.

When the Django 1.8 upgrade was done, all existing migrations were consolidated into “0001_initial” migration files. That consolidation was done on the master branch, not on the Cypress branch, so it included any migrations added after Cypress. The certificates app contained migrations like this one which added new tables - but these migrations were added post-Cypress. So, the end result is that migrations fail if you try to upgrade from cypress to dogwood directly (without performing the intermediate steps mentioned above). Django sees that some tables in the initial migration (like “certificatetemplate”) don’t exist yet, so it ignores the fake-initial request and tries to run the initial migration, which then fails, because most of the tables do exist.

You can’t import data directly from cypress to dogwood. You have to migrate it by following the steps outlined above. There’s a script in the configuration repo that does this for you, however I wouldn’t recommend running it on a production site without testing it thoroughly on a copy of your data first: https://github.com/edx/configuration/blob/adae6cf39c06ff5c4e242668c8dcba2b6c25d179/util/vagrant/migrate.sh

  • 显示引用文字 -

    Matjaz Gregoric
    @OpenCraft

Powered by eduStack & ifLab