In case anyone was wondering, yes we were down for ~2 hours or so. I apologize for the inconvenience.

We had a botched upgrade path from 0.17.4 -> 0.18.0. I spent some time debugging but eventually gave up and restored a snapshot (taken on Saturday Jun 24, 2023 @ 11:00 UTC).

We’ll likely stick to 0.17.4 till I can figure out a safe path to upgrade to a bigger (and up-to-date) instance and carry over all the user data. Any help/advice welcome. Hopefully this doesn’t occur again!

  • Aloso@lemmyrs.org
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    thank you for maintaining this server, and being very transparent about it ❤️

  • pohart@lemmyrs.org
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    I suspect almost everyone here understands how this happens and feels more sympathy than anything. We’ve been there, and with higher stakes. Thanks for taking the time to administer this instance.

  • lemmyrs@lemmyrs.orgOPM
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    Sigh, I tried it one more time. Same problem. Downtime this time was probably < 5 mins though so hopefully not noticeable. I am wondering whether the database is in some bad shape (no idea why/how). I guess we wait and retry with 0.18.1.