Since Microsoft released the re-write of Release Management in the form of the Release hub in VSTS, I've been dealing with a lot of legacy migrations and implementations of new release definitions. Based on what I've been seeing, I've arrived at a conclusion:
Release definitions should serve as deployment orchestrators, not as the deployment mechanism itself.
What do I mean by this? Simple, really. When creating a release definition, you have the option to build out your deployment any way you want from the in-the-box tasks, from tasks available on the marketplace, and from tasks you write and manage yourself. You also have the ability to write and run any script your heart desires, as long as you can figure out a way to get it into your release artifacts. Using a release definition as a deployment orchestrator means minimizing the number of deployment-activity related tasks you use, instead preferring to run scripts you write and manage yourself in order to perform your environment configuration and application deployment.
To put it as a very high level, the guideline is that the only tasks you should be using within a release definition are ones that either get binaries and deployment scripts pushed to a target machine (such as 'Windows Machine File Copy') and tasks that actually kick off a deployment script or take a direct deployment action ('PowerShell on Target Machines', or one of the tasks for DACPAC deployment or WebDeploy).
Environment-specific configuration values should be stored in source control and applied to your configuration files via your deployment scripts, at the time of deployment, instead of coming out of release or environment-level variables.
I'll give you 5 good reasons why.
- Versioning
- Transparency
- Testability
- Maintainability
- Auditability
Versioning
Simply put, we want to have one place to find everything about our application. Not just the code, but also anything responsible for provisioning environments (infrastructure as code), configuring environments (configuration as code), and deployment of the application.
Release definitions are versioned... separately from your application code. That is exactly the opposite of what we want! If the question is, "How was this version of the application deployed?", the way you answer that question should be "look at the build output, it's all in there." It shouldn't be, "Look at the build output, then figure out what releases were done from that build, then find the release that made it to the stage you're troubleshooting, then click around through a bunch of tabs to look at different values". Even if you see something that looks weird, you're not going to be able to test it, since you can't test a deployment without creating a release. More on that in a bit.
As your applications grow and evolve, your release definitions will, as well. By using an excessive number of release tasks to perform your deployment, you risk losing out on the ability to understand and replicate how older versions of your software were deployed, and you lose the ability to deploy an older version of your application using the exact same method and configuration parameters that it was originally deployed with.
Transparency
Tasks are a black box. You use them, but you don't know exactly what they do. The code for the Microsoft-provided tasks is on GitHub, but you have to know that in the first place. The exact usage of the tasks is locked up in the release definition itself, far away from the application code that is being deployed.
I always try to think of on-boarding a new developer. If they have to be told to look in a bunch of different places to understand how the application builds and deploys, we have a transparency problem. We want to minimize the number of different systems that contain critical application information, and maximize the transparency of those systems. Everything else a developer is doing is in source control... why would we keep critical information about our deployment process separate?
Testability
If a task fails, you can't easily hook a debugger up to it and step through the code. You can't even necessarily easily replicate the inputs or state. The only way to troubleshoot a failing task, really, is to rerun the release and hope for the best. The answer to the question, "Is this change going to break my deployment process?" should never be, "I don't know, let's see what happens when we run it."
Maintainability
The old RM server had a similar problem: Big, complex release definitions are hard to maintain. The more tasks you have, the more copy-and-paste you'll be going through, the harder it gets to reorganize, and the harder it becomes to have a mental model of exactly what your deployments are doing. The configuration variable management is limited enough to be problematic. It's impossible to reorganize configuration variables, you can't copy and paste the key/value pairs easily, you can't search and replace them easily, etc. Task groups and variable groups start to help mitigate this problem, but it's still not great, especially since variable groups can't (at the time of this writing) be applied at an environment level. A release definition with more than 5 or 6 tasks in a given environment pretty quickly becomes a big pain to maintain, and the more granular your tasks are, the worse it gets.
Auditability
Who changed the deployment process this weekend? Why are these configuration values incorrect, and how long have they been incorrect for? What values did we release this to production with last summer? All of these questions are trivially answered if the source for your deployment is source control. All of these are much more difficult to answer if you have to rely on the limited auditing/diffing capabilities built into the release definition editor.
Final words
Now, with all of that said, there are exceptions.
If you have a really simple application that doesn't need the additional complexity of following this approach, then don't do it! Do whatever is fastest and easiest!
If you're deploying to non-infrastructure hosting (such as containers or Azure Web Apps), different practices may apply. I'm hoping to coalesce my thoughts around those types of scenarios a bit and write a more in-depth blog.
