Are you upgrading from previous versions of TFS to TFS 2012 or setting up new TFS 2012?
Have you configured TFS app tier with HTTPS/SSL?
And when you configure the TFS 2012 build controller/agents, do you see Build Controller, Agents stop immediately after they start?
And does 'Details" hyperlink of the Build service shows HTTP code 404: Not Found
If the answer is yes for the above scenario, then, you need to relook at your steps and ensure you followed a critical change in 2012.
This link shows all steps to setup TFS with Https/SSL.
Assuming that you have acquired and installed the certificates, configured firewall, configured bindings in IIS.. still, if you are stuck with this error, probable solution is to pay attention to the manual settings to be done on the 'Message Queue' web.config change. This wasn't required in earlier versions and new in TFS 2012.
Navigate to the above link and goto "Configuring your Deployment" section, steps 8,9,10 and 11 the critical points for your problem. Below is a copy of those specific steps.
All the best !!
(copy paste from MSDN library)
Open the web.config file for Team Foundation Server in any plain-text editor, such as Notepad. By default, this file is located in the following directory: %PROGRAMFILES%\Microsoft Team Foundation Server 2012\Application Tier\Message Queue\web.config.
Remove the following endpoints from the file:
<!-- An empty relative address means the endpoint will pick up the base address of the svc file -->
<endpoint address=""
binding="customBinding"
bindingConfiguration="TfsSoapBinding"
contract="Microsoft.TeamFoundation.Framework.Server.WebServices.MessageQueueWebService" />
<!-- An empty relative address means the endpoint will pick up the base address of the svc file -->
<endpoint address=""
binding="customBinding"
bindingConfiguration="TfsSoapBinding"
contract="Microsoft.TeamFoundation.Framework.Server.WebServices.MessageQueueWebService2" />
Navigate to the <system.serviceModel> section, and add the following protocol mappings:
<protocolMapping>
<remove scheme="http" />
<add scheme="http" binding="customBinding" bindingConfiguration="TfsSoapBinding"/>
<add scheme="https" binding="customBinding" bindingConfiguration="TfsSoapBindingHttps"/>
</protocolMapping>
In the same <system.serviceModel> section, add the following binding to <customBinding>:
<binding name="TfsSoapBindingHttps">
<textMessageEncoding messageVersion="Soap12WSAddressing10" />
<httpsTransport authenticationScheme="IntegratedWindowsAuthentication" manualAddressing="true" />
</binding>
