March 4, 2026 at 12:00 am
Comments posted to this topic are about the item Rollback vs. Roll Forward
March 4, 2026 at 9:04 am
From the safety of my armchair, with no current emergency to disturb my reasoning, I would say: Get the DB to a Known State. And this will almost always be by rolling back. That's what ACID and database integrity are there for.
I don't want auditors and regulators throwing the book at me.
But, I suspect It Depends. I'd love to hear the reasoning for a opposing view.
MarkD
March 4, 2026 at 9:22 am
This was removed by the editor as SPAM
March 4, 2026 at 9:23 am
This was removed by the editor as SPAM
March 4, 2026 at 2:48 pm
I think fix-forward is the more common option when the issue has gone undetected for a period of time so that further processing has taken place that would be difficult to unpick (or lost if the database had to be restored from a backup). If it were possible to restore individual tables, this might be less necessary, but I wouldn't want to lose several hours of user input in an unrelated part of the database because a deployment in one area had gone wrong.
March 4, 2026 at 3:42 pm
From the safety of my armchair, with no current emergency to disturb my reasoning, I would say: Get the DB to a Known State. And this will almost always be by rolling back. That's what ACID and database integrity are there for.
I don't want auditors and regulators throwing the book at me.
But, I suspect It Depends. I'd love to hear the reasoning for an opposing view.
MarkD
I tend to agree, but it's not always easy. If data has changed, then ACID/integrity can be tricky. I don't think auditors necessarily are upset about rolling forward. It's more a question of knowing what you did and why.
March 4, 2026 at 3:43 pm
I think fix-forward is the more common option when the issue has gone undetected for a period of time so that further processing has taken place that would be difficult to unpick (or lost if the database had to be restored from a backup). If it were possible to restore individual tables, this might be less necessary, but I wouldn't want to lose several hours of user input in an unrelated part of the database because a deployment in one area had gone wrong.
Or days.
I have a lot of customers who don't find deployment issues for a few days.
March 4, 2026 at 3:56 pm
Steve, I admit I dashed off my response to your question before watching your delightful panel discussion. I have a slight feeling of having put my foot in my mouth.
It certainly does depend!
March 4, 2026 at 5:54 pm
Backward or forward? I would say it depends.
I think the most important skill for us DBA's, maybe even more than technical, is critical thinking under pressure. The ability to think 'big picture' and quickly understand the implications of our actions is to me the most important skill.
Rick
If you do a half-assed job of things, folks will ask 'why did this ass only do half the job?'
March 5, 2026 at 1:29 am
As a DBA I prefer rollback if the problem is discovered during deployment and fix the issue and re-deploy during the deployment window. If the problem can’t be fixed quickly, the deployment fail and should be rolled back. Some managers/leads don’t want 100% failure on the deployment so they push for partial success. A few days later when partial deployment or full deployment has problems, it is going to be a tough fix after end users start changing data. If this becomes a recurring pattern, major revisions need to be made to the design/develop/test process before proceeding.
March 5, 2026 at 3:10 pm
Just to add to hmbacon's reply, sometimes the problem isn't found during the deployment. I've had problems found days or weeks after deployment. Rolling back in those scenarios can be very challenging depending on what changed. If you update table structure and stored procedures and a different team deployed a new version of an app, that's a lot of stuff to roll back with potential for data loss in the process and a roll forward or an "emergency patch fix" may need to be deployed. In the above example, you could make the new columns NULLable and then roll back the SP, but can the application team roll back the app?
Deployments can get complex quickly when multiple teams are involved and rollbacks may be required or it may be better to roll forward. Plus, I've had deployments where it deployed smoothly but problems were found weeks later due to my deployment. Weird edge cases that got missed during testing that didn't exist in the old version. In those cases, the amount of data changed makes a rollback very difficult and a roll forward is the better option (in my job... may not apply to everyone).
The above is all just my opinion on what you should do.
As with all advice you find on a random internet forum - you shouldn't blindly follow it. Always test on a test server to see if there is negative side effects before making changes to live!
I recommend you NEVER run "random code" you found online on any system you care about UNLESS you understand and can verify the code OR you don't care if the code trashes your system.
March 5, 2026 at 5:34 pm
Backward or forward? I would say it depends.
I think the most important skill for us DBA's, maybe even more than technical, is critical thinking under pressure. The ability to think 'big picture' and quickly understand the implications of our actions is to me the most important skill.
For sure. This is a critical skill
March 5, 2026 at 5:36 pm
I tend to see with most customers that problems aren't discovered during deployment, but some time later (minutes, hours, days).
If there is a problem during deployment, most of the time it's a simple typo thing that someone let slip through without proper testing.
I do also see that in those cases, people tend to try and roll forward because so many deployments aren't for a single feature/changes, but multiple un-related ones, so there is a desire to not roll everything back
Viewing 13 posts - 1 through 13 (of 13 total)
You must be logged in to reply to this topic. Login to reply