Mailing List Archive: MythTV: Users

What to do if your upgrade fails (was Re: 0.22 database schema upgrade fails driver not loaded)

Index | Next | Previous | View Threaded

mtdean at thirdcontact

Nov 5, 2009, 7:19 PM

Post #1 of 5 (3781 views)
Permalink

What to do if your upgrade fails (was Re: 0.22 database schema upgrade fails driver not loaded)

On 11/05/2009 07:34 PM, Chris Tepe wrote:
> Michael T. Dean wrote:
>>> On 11/05/2009 05:26 PM, Chris Tepe wrote:
>>>> If I drop the old database and start with a blank database
>>>> everything appears to be working. The issue occurs when upgrading
>>>> the old database. If I can't upgrade the old database how do
>>>> import my previous settings? Or can the upgrade be done off line?
>>> Any chance you could http://filebin.ca/ (or post elsewhere) a
>>> gzip'ed or bzip2'ed copy of your 0.21-fixes (or whatever pre-upgrade
>>> version you were using) database backup for me to check out to see
>>> if we have an unhandled issue in the upgrades along the way?
> Her you go
> Your file is available at

After looking at Chris's backup, I found that the schema was modified in
such a way that it causes problems that prevent the DB upgrade from
succeeding. The solution here is to do a partial restore and only keep
the "non-re-createable" data. With this solution, you will save all
your recording metadata, your recording rules, and your recording
history. All other data can be re-created.

Unfortunately, a partial restore can only be done with the same DB
schema version. Therefore, to do this prior to an upgrade, you'll need
a blank 0.21-fixes DB into which to restore the portion of your
corrupted database. You can get one using 0.21-fixes mythtv-setup,
but--since you've already found out your upgrade to 0.22 failed--you
probably don't have 0.21-fixes installed, anymore.

Therefore, attached is a copy of a "blank" 0.21-fixes database backup.
(Yes, it has some data--that data is all required data created when the
schema is created and is not host-specific, so don't delete it.)

Note that this "blank" database backup will *ONLY* work with a 1214 or
1215 database backup. 1214 is MythTV 0.21-fixes and 1215 is just before
the database characterset conversion--i.e. what you'll likely have if
your upgrade fails and you forgot to do a real backup before upgrading
or some script helpfully deleted your 1214 backup.

Save the backup to your HOME directory and verify that you have the
right one (see the md5sum and sha1sum, below), then follow the steps
below to upgrade your database:

1) Back up your database (
http://www.mythtv.org/wiki/Database_Backup_and_Restore )

mythconverg_backup.pl

2) DROP your old database:

cat << "EOF" | mysql -umythtv -p mythconverg
DROP DATABASE IF EXISTS mythconverg;
CREATE DATABASE IF NOT EXISTS mythconverg;
ALTER DATABASE mythconverg DEFAULT CHARACTER SET latin1;
EOF

(and, no, the "latin1" is not a typo or a mistake, and, no, the "latin1"
won't cause any issues for users with non-latin characters).

3) Do a complete restore of the blank MythTV 0.21-fixes database:

mythconverg_restore.pl --dir=$HOME --file=mythconverg-1214-blank.sql
--verbose

4) Do a partial restore of your backup (fix the filename and directory
as appropriate):

mythconverg_restore.pl --dir=$HOME
--file=mythconverg-1214-20091105184542.sql.gz --verbose

5) Run the 0.22rc or 0.22-fixes or trunk mythtv-setup . It should
successfully upgrade your database. You will then have to reconfigure
MythTV (all of mythtv-setup configuration and all of mythfrontend
settings) and your plugins. (And, on the bright side, having to redo
your settings means that you will get to go through all the settings
that have been added since you first configured and set them the way you
really want them. :)

Good luck,
Mike

$ md5sum mythconverg-1214-blank.sql.gz
3684bdc9b80420aa965ac51211dacd41 mythconverg-1214-blank.sql.gz
$ sha1sum mythconverg-1214-blank.sql.gz
d966269cdfdb8b4a8991b65bf68b1048d2e2107e mythconverg-1214-blank.sql.gz

Attachments:

mythconverg-1214-blank.sql.gz (7.34 KB)

mtdean at thirdcontact

Nov 5, 2009, 8:59 PM

Post #2 of 5 (3673 views)
Permalink

Re: What to do if your upgrade fails (was Re: 0.22 database schema upgrade fails driver not loaded) [In reply to]

On 11/05/2009 10:19 PM, Michael T. Dean wrote:
> After looking at Chris's backup, I found that the schema was modified
> in such a way that it causes problems that prevent the DB upgrade from
> succeeding.

On deeper inspection, it seems that the relevant "modification" is
missing tables. The tables are mostly missing due to some corruption or
DB issue.

Though I haven't found/fixed the bug, yet, it seems that there's a bug
in Qt-MySQL drivers that kills the QSqlDatabase if a query attempts to
touch a non-existent table or column--thus the error message (from his
original post):

>> 2009-11-05 13:37:05.121 Newest MythTV Schema Version : 1244
>> QSqlQuery::exec: database not open
>> 2009-11-05 13:37:09.922 DB Error (Unable to perform test for database
>> corruption before character set conversion.):
>> Query was:
>>
>> Driver error was [1/-1]:
>> Driver not loaded
>> Database error was:
>> Driver not loaded
>>
>> 2009-11-05 13:37:09.923 Database Schema upgrade FAILED, unlocking.
>> 2009-11-05 13:37:09.924 Couldn't upgrade database to new schema

I'm looking into it to see if I can fix the bug for Qt, but I don't
think there's anything at all (that's sane that) we could do to work
around the bug in Myth code. Fortunately, (in theory) this bug can only
be tripped with a broken schema (as all of the code in Myth should only
be hitting real tables/columns).

Mike
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

cstepe at ctdataservices

Nov 6, 2009, 5:00 AM

Post #3 of 5 (3679 views)
Permalink

Re: What to do if your upgrade fails (was Re: 0.22 database schema upgrade fails driver not loaded) [In reply to]

Michael T. Dean wrote:
> On 11/05/2009 10:19 PM, Michael T. Dean wrote:
>> After looking at Chris's backup, I found that the schema was modified
>> in such a way that it causes problems that prevent the DB upgrade
>> from succeeding.
>
> On deeper inspection, it seems that the relevant "modification" is
> missing tables. The tables are mostly missing due to some corruption
> or DB issue.
>
> Though I haven't found/fixed the bug, yet, it seems that there's a bug
> in Qt-MySQL drivers that kills the QSqlDatabase if a query attempts to
> touch a non-existent table or column--thus the error message (from his
> original post):
>
>>> 2009-11-05 13:37:05.121 Newest MythTV Schema Version : 1244
>>> QSqlQuery::exec: database not open
>>> 2009-11-05 13:37:09.922 DB Error (Unable to perform test for
>>> database corruption before character set conversion.):
>>> Query was:
>>>
>>> Driver error was [1/-1]:
>>> Driver not loaded
>>> Database error was:
>>> Driver not loaded
>>>
>>> 2009-11-05 13:37:09.923 Database Schema upgrade FAILED, unlocking.
>>> 2009-11-05 13:37:09.924 Couldn't upgrade database to new schema
>
> I'm looking into it to see if I can fix the bug for Qt, but I don't
> think there's anything at all (that's sane that) we could do to work
> around the bug in Myth code. Fortunately, (in theory) this bug can
> only be tripped with a broken schema (as all of the code in Myth
> should only be hitting real tables/columns).
>
> Mike
> _______________________________________________
> mythtv-users mailing list
> mythtv-users [at] mythtv
> http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Mike,

Thanks for all the info. I'll give the partial restore a try as soon as
I can. Last night I removed .022 and reinstalled .21 using the backup I
created before upgrading:). So I have my complete 1214 database that I
had backed up. I'm curious as to what happened to the DB. I've been
using myth since 12/04 version 0.14 was my first. I upgrade that
through 0.17. I then built a new machine using Mythdora 4.0 then updated
the for the schedules direct (0.20). Since I prefer to wait a while
between updates I decided to switch my backend to CentOS for a longer
run between OS installs. I copied my database from my MythDora box,
installed 0.21 and was of to the races(er recordings). Which brings us
to my 0.22 upgrade. I've never modified any of the DB structure
myself so I'm not sure what happened.

I'll let you know how the partial restore goes. I believe i had to do
this when I moved from MythDora to my CentOS install.

Chris
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

mtdean at thirdcontact

Nov 6, 2009, 9:30 AM

Post #4 of 5 (3672 views)
Permalink

Re: What to do if your upgrade fails (was Re: 0.22 database schema upgrade fails driver not loaded) [In reply to]

On 11/05/2009 11:59 PM, Michael T. Dean wrote:
> On 11/05/2009 10:19 PM, Michael T. Dean wrote:
>> After looking at Chris's backup, I found that the schema was modified
>> in such a way that it causes problems that prevent the DB upgrade
>> from succeeding.
> On deeper inspection, it seems that the relevant "modification" is
> missing tables. The tables are mostly missing due to some corruption
> or DB issue.
>
> Though I haven't found/fixed the bug, yet, it seems that there's a bug
> in Qt-MySQL drivers that kills the QSqlDatabase if a query attempts to
> touch a non-existent table or column

FWIW, it's actually not killing our DB connection--just the error message.

> --thus the error message (from his original post):
>>> 2009-11-05 13:37:05.121 Newest MythTV Schema Version : 1244
>>> QSqlQuery::exec: database not open
>>> 2009-11-05 13:37:09.922 DB Error (Unable to perform test for
>>> database corruption before character set conversion.):
>>> Query was:
>>>
>>> Driver error was [1/-1]:
>>> Driver not loaded
>>> Database error was:
>>> Driver not loaded
>>>
>>> 2009-11-05 13:37:09.923 Database Schema upgrade FAILED, unlocking.
>>> 2009-11-05 13:37:09.924 Couldn't upgrade database to new schema
> I'm looking into it to see if I can fix the bug for Qt, but I don't
> think there's anything at all (that's sane that) we could do to work
> around the bug in Myth code.

And, now, it seems there may be something we can do in Myth to get a
better error message. Still have to figure out what.

Mike
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

cstepe at ctdataservices

Nov 8, 2009, 4:25 PM

Post #5 of 5 (3613 views)
Permalink

Re: What to do if your upgrade fails (was Re: 0.22 database schema upgrade fails driver not loaded) [In reply to]

Michael T. Dean wrote:
> On 11/05/2009 07:34 PM, Chris Tepe wrote:
>> Michael T. Dean wrote:
>>>> On 11/05/2009 05:26 PM, Chris Tepe wrote:
>>>>> If I drop the old database and start with a blank database
>>>>> everything appears to be working. The issue occurs when upgrading
>>>>> the old database. If I can't upgrade the old database how do
>>>>> import my previous settings? Or can the upgrade be done off line?
>>>> Any chance you could http://filebin.ca/ (or post elsewhere) a
>>>> gzip'ed or bzip2'ed copy of your 0.21-fixes (or whatever
>>>> pre-upgrade version you were using) database backup for me to check
>>>> out to see if we have an unhandled issue in the upgrades along the
>>>> way?
>> Her you go
>> Your file is available at
>
> After looking at Chris's backup, I found that the schema was modified
> in such a way that it causes problems that prevent the DB upgrade from
> succeeding. The solution here is to do a partial restore and only
> keep the "non-re-createable" data. With this solution, you will save
> all your recording metadata, your recording rules, and your recording
> history. All other data can be re-created.
>
> Unfortunately, a partial restore can only be done with the same DB
> schema version. Therefore, to do this prior to an upgrade, you'll
> need a blank 0.21-fixes DB into which to restore the portion of your
> corrupted database. You can get one using 0.21-fixes mythtv-setup,
> but--since you've already found out your upgrade to 0.22 failed--you
> probably don't have 0.21-fixes installed, anymore.
>
> Therefore, attached is a copy of a "blank" 0.21-fixes database
> backup. (Yes, it has some data--that data is all required data
> created when the schema is created and is not host-specific, so don't
> delete it.)
>
> Note that this "blank" database backup will *ONLY* work with a 1214 or
> 1215 database backup. 1214 is MythTV 0.21-fixes and 1215 is just
> before the database characterset conversion--i.e. what you'll likely
> have if your upgrade fails and you forgot to do a real backup before
> upgrading or some script helpfully deleted your 1214 backup.
>
> Save the backup to your HOME directory and verify that you have the
> right one (see the md5sum and sha1sum, below), then follow the steps
> below to upgrade your database:
>
> 1) Back up your database (
> http://www.mythtv.org/wiki/Database_Backup_and_Restore )
>
> mythconverg_backup.pl
>
>
> 2) DROP your old database:
>
> cat << "EOF" | mysql -umythtv -p mythconverg
> DROP DATABASE IF EXISTS mythconverg;
> CREATE DATABASE IF NOT EXISTS mythconverg;
> ALTER DATABASE mythconverg DEFAULT CHARACTER SET latin1;
> EOF
>
> (and, no, the "latin1" is not a typo or a mistake, and, no, the
> "latin1" won't cause any issues for users with non-latin characters).
>
> 3) Do a complete restore of the blank MythTV 0.21-fixes database:
>
> mythconverg_restore.pl --dir=$HOME --file=mythconverg-1214-blank.sql
> --verbose
>
> 4) Do a partial restore of your backup (fix the filename and directory
> as appropriate):
>
> mythconverg_restore.pl --dir=$HOME
> --file=mythconverg-1214-20091105184542.sql.gz --verbose
>
> 5) Run the 0.22rc or 0.22-fixes or trunk mythtv-setup . It should
> successfully upgrade your database. You will then have to reconfigure
> MythTV (all of mythtv-setup configuration and all of mythfrontend
> settings) and your plugins. (And, on the bright side, having to redo
> your settings means that you will get to go through all the settings
> that have been added since you first configured and set them the way
> you really want them. :)
>
> Good luck,
> Mike
>
>
> $ md5sum mythconverg-1214-blank.sql.gz
> 3684bdc9b80420aa965ac51211dacd41 mythconverg-1214-blank.sql.gz
> $ sha1sum mythconverg-1214-blank.sql.gz
> d966269cdfdb8b4a8991b65bf68b1048d2e2107e mythconverg-1214-blank.sql.gz
>
This fixed my database upgrade issues.
Thanks,
chris
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads