– Who is at fault?

Credits: Arsalan Khan, Ed Heironimus, Francis Wisseh, Maryam Moussavi, and Udhayakumar Parerikkal


This research paper analyzes the Center for Medicare and Medicaid (CMS)’s project in detail and makes recommendations on what could have been done differently. The project had 55 federal contractors working on it but this research paper will concentrate on only three. These federal contractors are:

  • CGI Federal who was developing and implementing the Federally-Funded Exchange (FFE). The estimated value of the contract was $93.7 million and it was awarded in December 2011.
  • Optum/QSSI was developing the Data Services Hub that would verify citizenship, immigration status and tax information. The estimated value of the contract was $144.6 million and it was awarded in January 2012. Optum/QSSI was also developing the Enterprise Identity Management (EIDM) that would provide enterprise-wide credentials and single sign-on capability. The estimated value of the contract was almost $110 million and it was awarded in June 2012.
  • Terremark Worldwide, In., (acquired by Verizon) was going to help increase CMS’ Platform-as-a-Service (PaaS) capabilities in the CMS cloud-computing environment. The total estimated value of the contract was $55.4 million and multiple task orders were issued until the summer of 2013.

The following tables summarize this research paper:

Table 1: Key Inputs

Key Inputs of the Project

CMSCGI FederalOptum/QSSI


·      Affordable Care Act

·      States

·      People/Team

·      FFE RFP

·      Requirements

·      Data Services Hub and EDIM RFP

·      PaaS RFP

Table 2: Key Components

Key Components of the Project

CMSCGI FederalOptum/QSSI


·      Agile Methodology

·      Project/System Integrator

·      Parallel “stacking” of phases

·      CMMI Level 5 Maturity

·      Agile Methodology

·      CMMI Level 3 Maturity

·      Agile Methodology

·      Data Services Hub Documents

·      EIDM Documents

·      Architecture diagram

·      Security

Table 3: Quality of Project Management – Qualitative View

Qualitative View of Project Management

CMSCGI FederalOptum/QSSI


·      Government vs. Private industry projects

·      Test plans and test reports

·      Requirement changes

·      Lessons learned from a state exchange

·      Requirement changes

·      Previous benchmarking and audits used

·      Issue escalation

·      Poor coordination

Table 4: Quality of Project Management – Quantitative View

Quantitative View of Project Management

CMSCGI FederalOptum/QSSI


·      HHS Enterprise Life Cycle·      Highly metrics-driven·      Use of charts·      Delayed processing of orders

Table 5: Project Management Successes and Failures

Key Successes and Failure of Project Management



·      Pressure from White House

·      Lack of business processes

·      Miscalculated costs

·      Various technical options were not considered

·      FFE

·      Changing Requirements

·      Testing

·      Data Services Hub and EIDM

·      Buggy Data Services Hub and EIDM

·      Financial Success

·      Hardware Outage

·      Project Management

Table 6: Lessons Learned

Lessons Learned in Project Management Best Practices

·      Roles and Responsibilities Matrix

·      Full-Time Project Manager

·      Business Processes and Governance

·      Requirements Management

·      Communications and Sharing

·      Metrics and Measurements

·      Methodologies and Documentation

Table 7: Recommendations

Team Recommendations

Project Design

Project Implementation

·      Project Manager

·      Cross-Functional Team

·      Team Satisfaction Survey

·      Pilot Approach


The Affordable Care Act (ACA) is the nation’s healthcare reform law enacted on March 23rd, 2010. Under the law, a new “Patient’s Bill of Rights” gives the American people the stability and flexibility they need to make informed choices about their health. There were numerous reasons why healthcare reform was critically needed in the United States, including:

  • High health insurance rates and lack of coverage by many: In 2013 the Congressional Budget Office (CBO) estimated that 57 million Americans under the age of 65 are uninsured; representing 1 out of 5 in the population.
  • Unsustainable healthcare spending: Healthcare spending represented 17.9% of the nation’s Gross Domestic Product (GDP) in 2011.
  • Lack of emphasis on prevention: 75% of healthcare dollars are spent on treating preventable diseases, however only 3 cents of each healthcare dollar goes toward prevention.
  • Healthcare disparities: Healthcare inequalities related to income and access to coverage exists across demographic and racial lines.

On October 1st, 2013 went live as part of the technical implementation of the ACA reform to help Americans buy healthcare insurance, however, the release was an astronomical failure. The cause and contributing factors that led to issues with this project are explored in detail. Focus is placed on looking at CMS’ capabilities from a Project Integration and Project Management perspective. Additionally, our analysis will assess the role of the major Federal contractors in the project. Examples are included to show how contributing factors such as scope creep, schedule constraints and lack of adequate testing led to a website that provided an inadequate customer experience.

This research paper provides a descriptive review and analysis of the project. During our analysis, we used Kathy Schwalbe’s (2014) Three-Sphere Model for System Management which entails organizational, business and technological perspectives of Project Management. We utilize these perspectives to determine what went wrong with the project from the Federal Government, CGI Federal (contractor), United Healthcare QSSI (contractor) and Verizon (contractor) points of view.   Furthermore, building on these unique perspectives, we analyzed the stated objectives and real implications of the project, the quality of the project management from a qualitative and quantitative perspective, key successes and failures factors, key lessons learned project management best practices, and recommendations for what might have been done differently.


In order to set the context of the research paper, we have to understand what the CMS does, what the reason for the website is, and why the on-time completion of the website was a priority. Additionally, we will look at the key inputs, components, and deliverables of this project.

3.1 About CMS

According to the and websites, CMS is one of the operational divisions of the Department of Human and Health Services (HHS). CMS is responsible for providing oversight on the Medicare and Medicaid program, Health Insurance Marketplace and related quality assurance activities. It has 10 regional offices with its headquarters in Maryland. The Administrator for the CMS is nominated by the US President and confirmed by the Senate. Figure 1 (Priorities, 2014) below shows CMS’s 2013 budget that accounted for 22% of the entire US Federal Government Budget.


Figure 1: Federal Budget Distribution

3.2 Why CMS was given the project?

CMS was designated as the chosen one by the Obama administration for obvious reasons. As discussed earlier the purpose of the ACA was to enable all Americans with the ability to buy health insurance. We also defined the CMS as an organization that provides healthcare to the elderly, disabled, and those who are not financially capable of buying healthcare. project began as a sub-agency under HHS called the Center for Consumer Information and Insurance Oversight (CCIIO). This office’s charter was to support a successful rollout of the ACA. In 2011, the secretary of HHS, Kathleen Sebelius “Citing efficiency gains…” stated that the CCIIO would be moved under CMS (Reichard, John. Washington Health Policy Week in Review Sebelius Shuffles Insurance Oversight Office into CMS, Shifts CLASS Act to Administration on Aging. January 10, 2011). The Obama administration insisted this was a way to control IT costs and leverage economies of scale through existing investments and infrastructure.   The Republican opposition believed this was another example of “…resources being diverted from seniors’ health care to be used to advance the Democrats’ new government-run health care entitlement” (Reichard, John. Washington Health Policy Week in Review Sebelius Shuffles Insurance Oversight Office into CMS, Shifts CLASS Act to Administration on Aging. January 10, 2011).

3.3 About the Federal Contractors and their relationship with the CMS

3.3.1 CGI Federal

CGI Group Inc., a Canadian company acquired American Management Systems (AMS) in 2004 to enter the U.S. Federal Government market. Due to this acquisition of an American Federal contractor by a foreign entity, there was a “firewall” created so that CGI Federal (formerly called AMS) could continue to work on Federal contracts. This “firewall” entailed that CGI Federal would not share Federal client information with CGI Group Inc. thus CGI Federal became a wholly-owned subsidiary of CGI Group Inc. This acquisition proved to be very lucrative for CGI Group Inc. CGI Federal became one of the most profitable business units due to the Healthcare and Human Services division. This division provides IT services in the areas of provider-based services, public health surveillance, portal integration, security, enterprise architecture, service-orientated architecture, business intelligence, and application development.

In September 2007, CGI Federal, among 16 other Federal contractors was awarded the Enterprise Systems Development (ESD) Indefinite Delivery Indefinite Quantity (IDIQ) contract by the CMS. The purpose of the ESD IDIQ was to support CMS’ Integrated IT Investment & Systems Life Cycle Framework and various IT modernization programs and initiatives. Although there were no task orders issued under this contract at the time it kept the door open for future task orders to the 16 contractors.

The project was competitively bid under the ESD IDIQ. This bid resulted in four contractors becoming the finalists. Out of those four contractors, CGI Federal was selected on September 2011 as the awardee based on “best value”.

3.3.2 Optum/QSSI

Founded in 1997, Quality Software Services Inc. (QSSI) is an established Capability Maturity Model Integration (CMMI) Level 3 organization with a proven track record of delivering a broad range of solutions with expertise in Health IT, Software Engineering, and Security & Privacy. Based in Columbia, MD Optum/QSSI is a subsidiary of UnitedHealth’s Optum division and was acquired by Optum in 2012.

Optum/QSSI is privately held with about 400 employees and collaborates with both the public sector and the private sector to maximize performance and create sustainable value for its customers. In its 15 year existence, the company has cultivated a process-driven, client-focused method of IT solution development, which, has solidified its reputation as a capable IT partner in both the federal and commercial marketplace. In the federal landscape, Optum/QSSI has established itself as an industry in the field of Health IT and this reputation was key in the company’s selection as a federal contractor for

3.3.3 Verizon

Terremark is now a Verizon company, dedicated to combining the strongest cloud-based platform with the security and professional services that are necessary to conduct today’s enterprise and public sector business across the next-generation IT infrastructure. At the center of Verizon’s capabilities is its enterprise-class IT platform, which combines advanced IT infrastructure with world-class provisioning and automation capabilities which is what CMS leveraged in this case. Verizon’s standards-based approach fully aligns with today’s enterprise business requirements driven by agility, productivity, and competitive advantage.

Verizon Terremark was a natural fit at the CMS through a long-standing relationship. Verizon manages and maintains the entire HHS Wide Area Network (WAN) along with ancillary services such as security services, mobile solutions, and unified communications. Verizon also developed a homegrown fraud detection service is used to identify toll-free fraud to pursue Medicare and Medicaid fraud saving the agency millions.

3.4 Key Inputs of CMS

For the project, we identified the following key inputs for CMS:

  • Patient Protection and Affordable Care Act (ACA): The ACA became law on March 23rd, 2010 under the Obama Administration. The legislation was passed to address various consumer health insurance purchase issues such as a denial of coverage due to pre-existing conditions, stopping of coverage if patients became sick, lifetime benefit limits and access to affordable healthcare. The ACA mandated the creation of “exchanges” that would be used by consumers to compare and buy a Qualified Health Plan (QHP) based on their state of residence, income level, age, and other factors. These exchanges could be created at the state level or at the federal level. If states decide not to create their own exchanges then they would have the option to redirect their constituents to the federal exchange to buy healthcare insurance.
  • States: The various states and Washington D.C. informed CMS if they intend to create their own exchanges or they would utilize the exchange developed by the Federal government. States also had the flexibility to utilize the federal exchange when they wanted and thus initially 26 states opted to have their constituents go to the federal exchange to purchase healthcare insurance.
  • People/Team: CMS assigned a part-time Project Manager to the project.

3.5 Key Inputs of for Federal Contractors

The project is one of the most complex systems in recent times. The project entailed 55 contractors working on various aspects of the system. These contractors were responsible for the creation of a robust network/infrastructure, the development of a website front-end, the Federally-Funded Exchange (FFE), the Data Hub and the EIDM. Additionally, this system receives eligibility and verification information from various other federal government agencies as the consumer fills out the online form. The following figure (Ariana Cha, 2013) below shows the complexity of the information flow of the entire system. contractors and agencies processes.png

Figure 2: contractors and agencies processes

3.5.1 CGI Federal

CGI Federal was one of the prime contractors for the project. It had the following key inputs:

  • Request for Proposal (RFP): An RFP was one of the first key inputs for the project. It required the establishment of a FFE that would be used for eligibility and enrollment, plan management, financial management, oversight, communication, and customer service. The following Figure (Desai, 2013) shows the FFE Concept of Operations:

Figure 3: FFE Concept of Operations

  • Requirements: After the contract was awarded, first the CCIIO and then various other representatives within CMS provided requirements to CGI Federal for the FFE. These representatives came from policy, legal and Office of Information Services (OIS).

3.5.2 Optum/QSSI

  • Request for Proposal (RFP): This was the primary input for the project. It required the development of a data services hub for information exchange and the EIDM for user account registration.

3.5.3 Verizon

  • Request for Proposal (RFP): One of the first RFPs released in 2010 was for the infrastructure and essentially Platform as a Service (PaaS) for the ACA as dictated in the 2010 RFP. A Cloud Solutions Executive for Verizon Terremark said Verizon received an award prior to the additional contractors involved. Following the award, CGI Federal and others were asked to develop the system to conform to the Verizon environment.

3.6 Key Components of for CMS

  • Systems Development Methodology: A presentation from April 2012 by CMS’ OIS shows that an “Agile” methodology was used for the as shown in the following figure (Services, 2012).

Figure 4: CMS Agile Methodology

  • Project/System Integrator: CMS took on the role of “system integrator” to manage all 55 contractors.
  • Implementation Consideration: A McKinsey report shows the parallel “stacking” of all phases for this project as shown below (CMS, Red Team Discussion Document, 2013):

Figure 5: McKinsey Report for CMS

3.7 Key Components of for Federal Contractors

3.7.1 CGI Federal

  • Process Methodology: Patrick Thibodeau indicates in a Computerworld article that CGI Federal attained Capability Maturity Model Integration (CMMI) Level 5 maturity making it only the 10th company in the US to achieve this level. By extension, we can assume that CGI Federal brought the best practices of CMMI for the project.
  • Systems Development Methodology: Based on Federal contracting experience, the Federal contractor would either have their own development methodology or they would use the development methodology of the client (CMS in this case). Research indicates that CGI Federal used an Agile methodology to develop the FFE.

3.7.2 Optum/QSSI

  • Process Methodology: As a CMMI Level 3 organization, Optum/QSSI has a reputation for process-driven and client-focused methods of IT solutions development. Based on our research it is evident that the company implemented CMMI best practices on the project.
  • Systems Development Methodology: According to a senior analytics consultant with a major health provider who worked on the project, Optum/QSSI used an agile development methodology similar to the one depicted below (Group, 2014) based on iterative and incremental development with continuous visibility and opportunity for feedback from CMS.

Figure 6: Agile Methodology

  • Requirements Documentation for Data Services Hub: The Data Services Hub is a central function of the federal exchange which connects and routes information among trusted data sources including Treasury, Equifax, Social Security Administration (SSA), etc. Inputs from CMS which changed in late September 2013 to allow for an account creation before shopping for health plans.
  • Requirements Documentation for EIDM: EIDM enables healthcare providers to have the ability to use one credential to access multiple applications, serving the identity management needs of new and legacy systems. Inputs from CMS which changed in late September 2013 to allow for an account creation before shopping for health plans.

3.7.3 Verizon

  • System Architecture Design: There were an architecture diagram and overall design for the entire system that lost effectiveness due to a lack of accountability to ensure each component was delivered.
  • Security: Security was a huge component within the requirements for infrastructure, and Verizon Terremark offered a highly secure architecture designed to meet all of the critical compliance and certification requirements. Verizon had been audited against FISMA to the moderate-level and NIST 800-53 for federal customers. Verizon was also asked for advanced security options on the platform such as intrusion detection/intrusion prevention (IDS/IPS), log aggregation, and security event management.

3.8 Key Deliverables for CMS

  • Website: A website that should be able to provide residents the ability to compare QHPs.
  • Exchange: A website that should enroll residents by verifying their eligibility based on income level, age, and other factors.

3.9 Key Deliverables for Federal Contractors

3.9.1 CGI Federal

  • FFE: A fully functional FFE would be ready to go live by October 1st, 2013. The FFE would be the backbone of and would seamlessly integrate with the website, the data hub, and the EIDM.

3.9.2 Optum/QSSI

  • Data Services Hub: This system of determines eligibility for financial help. It sends customer data to various government agencies (VA, DHS, Treasury, etc.) to verify eligibility.
  • EIDM (Proof of Identity): Upon account creation, this system verifies identity with Experian. The system also enables healthcare providers to have the ability to use one credential to access multiple applications, serving the identity management needs of new and legacy systems.

3.9.3 Verizon

  • PaaS: Fully operational infrastructure which provides servers and hosting for the exchange.
  • Environmental: Supports power, connectivity, and memory requirements for the environment.
  • Service Level Agreement (SLA): Rolling out the infrastructure in a timely fashion, offering and executing upon SLA’s required by the Government among other things.


4.1 Project Management Quality of CMS

4.1.1 Qualitative

  • Quality Planning: Quality planning for government releases is at a different scale than quality planning for private companies.  Many factors come into play such as redistributing of the resources through regulation, subsidization, and procurement. As part of CMS’ quality planning phase, the main scope aspects were functionality, features, and system outputs.  However, performance, reliability, and maintainability suffered heavily due to time constraints as October 1st, 2013 was the hard deadline.
  • Quality Assurance: CMS used test plans and test reports to ensure quality coverage were being met as per the requirements.  The front-end web interface was indeed completed in time.  However, identifying quality system integration was difficult due to the complexity of the back-end sub-systems.

4.1.2 Quantitative

  • Quality Monitor and Control: During the implementation phase, CMS didn’t take proactive measures to address the issues they found one week before launch. Specifically, when the testers reported server crashes at a scale of 10,000 concurrent users. Additionally, CGI Federal had reported more testing is required yet it appeared that CMS was insensitive to their recommendations.  Status reports were supposed to be read, understood and acted upon. HHS followed the Enterprise Life Cycle and CMS was supposed to follow these guidelines.

4.2 Project Management Quality for Federal Contractors

4.2.1 Project Management Quality of CGI Federal Qualitative

  • Quality Planning: As a CMMI Level 5 organization CGI Federal had optimized quality processes to deliver appropriate outcomes for the FFE. However, requirement changes seem to be one of the main issues with the project. Requirements were still being revised until the summer of 2013 and kept on evolving even a week before go-live. Additionally, the number of states that were going to join FFE increased from 26 to 34 which created another level of complexity in terms of maintaining the quality on the project.
  • Quality Assurance: According to Cheryl Campbell, Senior Vice President at CGI Federal, in the Congressional hearings CGI Federal developed the FFE as per the contract requirements. It is interesting to note that CGI Federal was also one of the companies that developed the Massachusetts Health Exchange that was used as a model for the FFE. Hence, we can make the assumption that quality lessons were learned from that project which could have been used for the FFE. Quantitative

  • Quality Monitor and Control: CGI Federal is a highly metrics-driven organization. Each project is monitored and measured according to industry “best practices” and proprietary methodologies. Projects are evaluated based on scope, cost, schedule, and other factors to check the health of the project and verify if they are keeping the customer satisfied. But if the requirements continue to evolve then even the best methodologies and measurements are not a match for customers changing their minds.

4.2.2 Project Management Quality of Optum/QSSI Qualitative

  • Quality Planning: As a CMMI Level 3 organization Optum/QSSI had a planned quality process to deliver appropriate outcomes for the Data Services Hub and EIDM project deliverables. However, changing project requirements from CMS severely impacted quality planning efforts. For instance, the late requirement change in September that required consumers to create user accounts before browsing the exchange marketplace resulted in higher than expected simultaneous system usage and as a result impacted the functionalities of the EIDM tool which was originally designed to allow consumers to first access the systems, browse the marketplace, and if they wanted a product, create an account. Because the EIDM is only one tool for the federal marketplace registration system, this late requirement change made it impossible to coordinate and plan quality processes with other contractors who worked on portions of the registration system to ensure the appropriate performance outcomes before the go-live date of October, 1st.
  • Quality Assurance: Both the Data Services Hub and EIDM deliverables met quality assurance satisfying CMS’ requirements and all relevant quality standards for the project according to Andrew Slavitt’s, Group Executive Vice President at Optum/QSSI, during his Congressional hearings. It is important to also note that Optum/QSSI developed an EIDM tool for two other CMS systems. This EIDM tool followed benchmarking and quality audits taken from those existing EIDM solutions at CMS. Quantitative

  • Quality Monitor and Control: Requirements changes greatly impacted quality monitoring and control. Although Optum/QSSI used quality control tools such as charts to guide acceptance decisions, rework, and process adjustments, changing requirements severely impacted these controls. These changes introduced time constraint challenges and limited system-wide testing, and most importantly user acceptance testing.

4.2.3 Project Management Quality of Verizon Quantitative

  • Extensive delays in processing orders for additional capacity, provisioning resources, and implementation also caused Verizon a lot of angst with the CMS customer. Qualitative

  • Management within Verizon also failed to run some of the concerns up the executive flagpole to make leadership aware of issues that could have prevented delays or numerous escalations by CMS.
  • Verizon’s project management failed on many accounts. Poor coordination was to blame between multiple project managers assigned to the project within.

4.3 Project Management Key Successes and Failures

4.3.1 CMS

If we review the congressional hearings and documentation, they reveal that the project was a high priority project for CMS. In conversations with Federal contractors, CMS would start by saying “this is what the White House wants…”. It is still unclear whether this prefix was used because directions were actually coming from the White House or whether it was just used to indicate the importance of the project. Regardless of the intentions, one thing is for sure that they were not followed by action since there was not a dedicated full-time Project Manager to manage the project from kickoff to implementation. Most likely decisions were made by committees as is often the case with large government projects.

A big piece of the project included behind the scenes business processes even before the technology was to be considered. These business processes and governance entailed not only coordinating with 26 states but also with insurance companies. The picture below depicts an exhaustive list of stakeholders that were affected by the project:

FFE Stakeholders.png

Figure 7: FFE Stakeholders

From a business standpoint, CMS failed to calculate in advance the true cost of the entire project. Additionally, even after reports by McKinsey indicating a danger of not doing end-to-end testing and warnings from CGI Federal in their August 2013 status report (Federal, 2013) that testing could be an issue, CMS ignored these experts and went full steam in going live on October 1st, 2013.

Research indicates that some COTS products and custom software were developed to standup It also seems like CMS failed to look at the various internal and external “firewalls” the system needed to pass through.

4.3.2 Federal Contractors CGI Federal

  • FFE: According to Congressional hearings, the CGI Federal representative indicated that they had provided a fully functioning FFE as per contract requirements by October 1st, 2013. This was their success factor.
  • Changing Requirements: CGI Federal was responsible for developing the FFE. It was put under the spotlight for not providing recommendations holistically for the entire project. It is evident that requirements were changing and new states were being added. But there was no push back from CGI Federal to indicate that the required changes would result in quality issues on their end that would affect the entire system.
  • Testing: While the system did work for an initial couple of users but logging delays resulted in poor customer experience. Research indicates that no end-to-end testing was performed to see holistically how the system would work. CGI Federal could have used their vast amount of industry expertise to inform CMS that no end-to-end testing would result in major issues. Optum/QSSI

  • Data Services Hub & EIDM: Based on Andrew Slavitt’s Congressional hearings on, Optum/QSSI successfully developed and delivered a fully functional Data Services Hub and EIDM tools. For example, according to Slavitt on October 1st the Data Services Hub processed over 175,000 transactions and millions more after the project launched.
  • Buggy Data Services Hub & EIDM: In the same Congressional hearings, however, Andrew Slavitt acknowledged that the Data Services Hub and EIDM tools, although worked functionally as designed, experienced performance bottlenecks when the project launched because of the late requirements changes requiring consumers to create accounts before browsing the marketplace. This change resulted in higher than expected simultaneous usage of the registration system and the Data Services Hub eligibility verification tool. Slavitt also admitted to the fact that Optum/QSSI identified and fixed bugs in the EIDM tool days after the October, 1st launch date. The release of code that had bugs was a quality failure and contradicts Slavitt’s earlier comments about delivering a fully functional EIDM tool. Verizon

  • Financial Success: The primary success story for Verizon was that financially the company did far better as a result of this project than initially predicted based in large part to the scope creep. Additionally, Verizon specifically was not the cause for delays or outages on day one of the projects and delivered the infrastructure to support the site by launch date. A significant underestimation of capacity would be to blame for the initial failures of
  • Hardware Outage: subsequent failure which Secretary Sebelius cites in her testimony was a hardware outage unrelated to project management (Krumboltz, Michael. suffers outage as Sebelius testifies that it never crashed.).
  • Project Management: Key project management failures within Verizon were inherent with the ordering, design or engineering phase, and eventually, implementation suffered due to project management inefficiencies. As discussed previously, Verizon had several members of a project management team supporting the CGI Federal relationship, QSSI, and CMS relationships. There should have been one central program manager who supported the ACA contract for Verizon. The communication failure through project management teams created bottleneck failures that spread throughout the engineering teams.

4.4 Lessons Learned on Project Management Best Practices

  • Who’s on First: Even though this project needed input from various internal and external stakeholders, clear roles and responsibilities matrix should have been developed. This matrix should have been used by the contractors to see who is responsible for various activities of this large project and who specifically reach to out to in case they ran into bottlenecks.
  • Project Manager: CMS did not assign a full-time Project Manager for the project. For this large-scale project, it would have been prudent to have a full-time project manager responsible for coordinating various activities internally and across various contractors.
  • Business Process and Governance: Since this was the first time such an endeavor was taking place with multiple stakeholders, it would have been useful to map the business processes of the future state prior to award. Overall business processes would also support governance structures that would help in checking progress and checking alignment with the stated objectives of the project.
  • Requirements Management: Ever-changing requirements and a change in strategy can affect projects dramatically. For, there should have been some baseline requirements established earlier in the project. The baseline requirements would entail the basic functionalities needed by CMS. It would have been useful to use a Requirements Traceability Matrix (RTM) that would be available to everyone on the project. The RTM would not only help all stakeholders be informed of what is going but it would keep the entire team honest as well and perhaps identify any issues before they become a problem later.
  • Communications and Sharing of Information: The way the project was set up it seems like the left-hand did not know what the right hand was doing. This can create problems in understanding issues from a holistic perspective and not knowing if there are dependencies that should be coordinated.
  • Metrics and Measurements: Reasonable metrics should have been created to assess the health of the project. These metrics should measure the stakeholder and team satisfaction at the beginning and during the project in the project life cycle to determine where adjustments need to be made. Based on these metrics, remediation processes should have been set up so that nothing falls through the cracks.
  • Methodologies and Documentation: Although it seems like CMS was supposed to follow HHS’ Enterprise Life Cycle (ELC) but research indicates that an Agile methodology was used to develop the system. This alludes to conflict in terms of what is supposed to be used versus what was actually being used. Additionally, the vendors who were helping CMS came with their own methodologies. It seems like there were too many methodologies but not a consistent alignment of them across the organization so that all teams were on the same page. In this scenario, the advice would be to understand the various methodologies at play and select the most appropriate one. This selection might also entail having some sort of a hybrid methodology that everyone conforms to. Having one methodology would reduce the amount of documentation that needs to be developed thus freeing up resources to work on the actual needs of the project.


5.1 Project Design Recommendations (CMS only)

  • Project Manager (PM): The project had 55 federal contractors working on it at various times. The system components that these federal contractors were developing were dependent on each other. For example, the website needed to communicate with EIDM and Data Hub which would communicate with FFE. Due to these complex dependencies, there was a significant amount of communication and coordination that needed to happen on this project. Additionally, someone needed to see how not only if these system components would work with each other but how the thorough testing of these systems would be the difference between a failed project versus a successful project. Despite the complexity of managing such a large group of contractors and understanding the pros and cons, CMS did not have a full-time PM for While the real reasons for this decision are unknown, we can extrapolate from research that lack of an astute full-time PM was one of the major causes of the issues with

We recommend that a full-time PM should have been assigned to who had experience in large-scale implementations. This PM would have the authority to push back on unrealistic timelines, have a holistic view of the project and understand that even though individual system components are being developed, there should be time allocated in the project to perform effective end-to-end testing.

  • Cross-Functional Teams: As discussed earlier, the system components that the federal contractors were developing had many dependencies on each other. Despite these dependencies, no proof has been found that cross-functional teams were established.

We recommend that in designing the project, the development of cross-functional teams should have been given a high priority. These cross-functioning teams would comprise of government and contractors. These teams would not only create synergies among the various people but should be designed in a way where sharing of lessons learned and recommendations are encouraged.

  • Team Satisfaction Survey (TSS): When a project falls off the track, oftentimes it is because once management pays attention to it, they are at a point of no return. This is what seems to have happened at CMS. By the time people working on the project started showing their concerns that testing could be an issue, CMS either ignored this or that they did not have a choice and thus went ahead to release a buggy version of the system to the masses.

We recommend that a TSS should be incorporated into the project whose purpose would be to ask people at all levels about their concerns and recommendations of the project. The TSS should collect this information at the beginning and periodically during the project. The TSS should also have a mechanism where actions could be taken promptly by management if there are issues that seem to be recurring. The TSS is not a status report but actually a mechanism to check the pulse of the project.

5.2 Project Implementation Recommendations (CMS only)

  • Pilot Approach: The project used a “big bang” approach to release the software on October 1st, 2013. This approach resulted in overwhelming the system as users who tried to login found that their online forms were either taking too long to verify their information or that simply they were kicked out of the system. Additionally, it is apparent that the federal government did not anticipate the system errors it was going to get when the system went live.

We recommend that a pilot program should have been created for one of the states. This pilot program would be used to see how the system would perform when it goes live and what kind of issues it might have once it is open to the masses. Lessons learned from this pilot program could have been used to provide a better customer experience once the system went live.


To summarize, it should come as no surprise to those familiar with IT projects that most IT projects fail. A recent Gartner user survey showed that, while large IT projects are more likely to fail than small projects, around half of all IT projects fail in part due to unrealistic requirements, technical complexities, integration responsibilities, aggressive schedules, and inadequate testing. Causes that were all related to the project which resulted in its failure. As outlined in this case analysis, these fundamental missteps were the contributing factors that led to issues with this project and ultimately its failure. Based on our analysis CMS did not have the Project Integration and Project Management know-how to manage such a major project. The Agency’s assignment of a part-time project manager to the project is evident that leadership did not fully understand the magnitude and importance of the project, and what it took to implement IT projects. Based on our recommendations, it is our hope that such project management missteps are avoided for IT project implementations in the future.


  1. Ariana Cha, L. S. (2013, 10 24). What went wrong with Retrieved 04 16, 2014, from washington post:
  2. CMS. (2012, 04 01). Health Insurance Exchanges. Office of Information Services, 25.
  3. CMS.GOV. (2014, 04 01). CMS covers 100 million people. Retrieved 05 01, 2014, from cms:
  4. Conerly, B. (2013, 07 16). ObamaCare’s Delays: Lessons For All Businesses About Project Management. Retrieved 04 21, 2014, from
  5. C-SPAN. (2013, 10 24). Implementation of Affordable Care Act. Retrieved 03 15, 2014, from
  6. Daconta, M. (2013, 11 01). Media got it wrong: failed despite agile practices . Retrieved 04 21, 2014, from gnc:
  7. Desai, C. (2013, 03 05). Federally Facilitated Exchange. Health and Compliance Programs, 44.
  8. DHHS. (2012, 06 2012). Guide to Enterprise Life Cycle Processes, Artifacts, and Reviews. Center of Medicare & Medicaid Services.
  9. Dudley, T. (2012, 06 10). The Affordable Care Act and Marketplace Overview. CMS Office of Public Engagement.
  10. Federal, C. (2013). Monthly Status Report –August 2013. CMS. MD: CGI Federal.
  11. Hardin, K. (2013, 11 13). rollout lesson: Push back on unrealistic launch dates. Retrieved 05 02, 2014, from techrepublic:
  12. Office, U. S. (n.d.). Patient Protection and Affordable Care Act. DC: GAO.
  13. Simon & Co., L. (2013, 12 04). Major Contracts to Implelment the Federal Marketplace and Support State Based Marketplaces. DC: Simon & Co.
  14. States, C. o. (2013). 113 Congress. DC: House of Representatives.
  15. Thibodeau, P. (2013, 12 30). The firm behind had top-notch credentials and it didn’t help. Retrieved 03 04, 2014, from computer world:
  16. Thompson, L. (2013, 12 03). Diagnosis: The Government Broke Every Rule Of Project Management. Retrieved 04 01, 2014, from forbes :
  17. Turnbull, J. (2013, 12 12). What The US Government Learned About Software From The Healthcare.Gov Failure (And Some Of Us Already Knew). Retrieved 05 02, 2014, from gaslight:
  18. Walker, R. L. (2013). Response of CGI Federal Inc. to Additional Questions for the Record and Member Requests for the Record. DC: Wiley Rein LLP.

Success! You're on the list.

How to select an Enterprise Architecture Framework?


This article provides in detail the elements of an Enterprise Architecture (EA) framework that would be selected and deployed at a fictional United States (US) Federal Government contractor called FedCon. FedCon is divided into 3 Business Units (BUs) that are focused on providing Management Consulting, Information Technology (IT) Consulting and Systems Integration (SI) Services in Healthcare, Environment, and Finance.

This article analyzes FedCon in terms of Strategies, Politics, Innovation, Culture and Execution (SPICE) as shown below:

StrategiesCEO, COO, and CIOBusiness-IT alignment
PoliticsBU SVPs, PMO and program/project managersTechnology products and services
InnovationEmployees directly interfacing with customersTechnology products and services
Culture PMO, HR and Accounting/FinanceLeverage the massive intellectual property
ExecutionAll employeesOrganizational performance

Based on the above, it is determined that The Open Group Architecture Framework (TOGAF) would be the appropriate EA framework at FedCon since (1) it is supported by multiple vendor tools (2) it is constantly being improved upon and (3) it has an Architecture Development Methodology (ADM) which can be used as a guide.


FedCon is a fictional 30-year-old large publically traded US Federal Government contractor that provides Management Consulting, IT Consulting, and SI services to civilian agencies. It has over 5,000 employees nationwide and it is structured into three Business Units (BUs). Each BU has domain expertise in Healthcare, Finance or Environmental information systems. This structure allows the BUs to work directly with the civilian agencies based on their missions. Each BU has its own account/business development (BD) team that reports to the BU Senior Vice President (SVP). The Program/Project Managers report to the BU SVP and provide status updates on programs/projects to the corporate Program Management Office (PMO). The PMO conducts weekly meetings to provide guidance on corporate standards, compliance, and general project templates.

Organizational Structure
Organizational Structure


Over the past couple of years, FedCon has lost 20% of its business. The CEO has been under pressure by the shareholders to turn the company around. Thus, the CEO hired a management consulting firm to determine what were the pain points within FedCon that were preventing it from staying competitive in the marketplace. The management consulting firm’s report revealed that due to inefficient business processes and outdated technologies FedCon’s BUs were not able to collaborate efficiently to manage business and technology changes. Based on these findings in the report, the CEO mandated the Chief Operation Officer (COO) and Chief Information Officer (CIO) to work together to find areas that they can improve in the next 12 months.


In order to address the CEO’s concerns, the COO and CIO came to the conclusion that in order to help FedCon create a disciplined approach to managing strategic intent and its execution they had to look into the field of Enterprise Architecture. Thus, the COO and CIO decided to standup an Enterprise Architecture Program Management Office (EAPMO) that would report directly to the CIO. Initially, the EAPMO is tasked with determining the high-level criterions to select a framework. This task includes providing elements of the framework to be used and how the framework would be applied within FedCon..

In this article, we assess the feasibility of an EA framework that can be used in FedCon by making observations about Strategies, Politics, Innovation, Culture and Execution (SPICE) factors. These factors would focus on understanding the people, processes, and technologies at FedCon to create an effective EAPMO.

SPICE Factors
SPICE Factors

Strategies at FedCon:

At FedCon, there are multiple levels of strategies that are developed. These strategies include: (1) the corporate strategy determined by the CEO, (2) the operational strategy determined by the CFO, COO, and CIO, (3) the BU strategy determined by the BU SVPs and (4) the PMO strategy determined by the PMO office. This is shown below.

Multiple Corporate Strategies
Multiple Corporate Strategies

As we can see from the above figure, each strategy layer addresses different domains for the various stakeholders. Even though these strategies are developed to increase the bottom line and decrease costs, they are created in isolation. Additionally, since each BU is somewhat autonomous it can create technology products and solutions for the civilian agencies that overlap with corporate products and solutions. This is a problem since not leveraging the corporate assets where applicable for client delivery can result in program/project delays and duplicative systems.

The primary strategic concerns in choosing an EA framework are:

  1. Stakeholders – CEO, COO and CIO are the strategic stakeholders and the executive sponsors of the EAPMO.
  2. Domain – Strategically, FedCon is interested in the alignment of business and IT operations and efficient processes.

FedCon has never stood up an EA practice and thus it would be wise to select an EA framework that could guide them in what to do and that it has been proven in the industry to be useful for organizations that are just starting out their EA journey. These high-level strategic criterions are fulfilled by The Open Group Architecture Framework (TOGAF) that provides an Architecture Development Methodology (ADM) as a step-by-step guide and includes how to do stakeholder management.

Politics at FedCon:

Generally, when we talk about politics in an organization we are referring to the negative connotations attached to it. But for our purposes, we will define politics to mean the formal power or informal power of an individual or group within an organization. The power exhibited by these individuals and groups can turn into obstacles or support to bring about organization-wide changes. In this sense, here we refer to formal power as the reporting structures while informal power refers to the influence yielded based upon the size of the BU, revenue generated by BU, the headcount of BU and close relationships of BU leadership with the executives.

At FedCon, even though the BU SVPs have the same title, they don’t have the same power. Taking this into account and the emphasis by the US Federal Government Executive Branch to focus on healthcare issues, the largest and most profitable among the FedCon’s BUs in the healthcare BU. Due to this reason, healthcare BU SVP has more informal power among its peers. This means that if the healthcare BU can be convinced of the merits of the EA practice then we can come one step closer to a FedCon-wide EA practice.

The primary political concerns in choosing an EA framework are:

  1. Stakeholders – BU SVPs, PMO and program/project managers are the political stakeholders. The BU SVPs have the formal power to bring change within their respective BUs. The PMO is a well-established office and it has visibility into the various kinds of projects and has informal power by pushing down changes to the project level within different BUs. Lastly, the program/project managers within BUs are stakeholders as well since they have to indoctrinate their teams on how EA can be used as leverage when developing client technology products and services.
  2. Domain – Politically, agreement, collaboration, and coordination across BUs and the corporate team seems to be the area of focus to rapidly bring technology products and services.

Due to the “friendly” competition among BUs to become bigger and yield more influence in FedCon, politics has to be carefully considered. Sometimes BUs are not willing to share if there are possible overlaps with what they are developing and what is already available in a different BU or at the corporate level. Convincing BUs to work together could be hard and caution has to be taken in which players to involve in the development of the EA practice. Additionally, there has to be some sort of collaboration between the EAPMO and PMO for lessons learned and organizational improvements. These high-level political criterions are also fulfilled by TOGAF where it recommends how Architecture Governance and Architecture Boards should be set up.

Innovation at FedCon:

Broadly speaking, innovation in organizations is disruptive, incremental or a combination of both. Disruptive innovation as described by the world-renowned management theorist Clay M. Christensen’s institute is such that it “transforms an existing market or sector by introducing simplicity, convenience, accessibility, and affordability where complication and high costs are the status quo.” This disruption can come in the form of unique business models, products and/or services that can give rise to new industries and improve existing industries. On the other end of the innovation spectrum, incremental innovation is where small changes are made to existing business models, products and services to improve existing industries.

Being a US Federal Government contractor, innovation at FedCon is mostly incremental since it tries to improve upon its existing products and services that are provided to the civilian agencies. FedCon accomplishes incremental innovation by obtaining customer feedback and assessing the competitive landscape. However, since BUs only focus on their own expertise, there are fewer opportunities for collaboration across BUs, which means technology products and services, are being developed without leveraging what already exists in the organization.

The primary innovation concerns in choosing an EA framework are:

  1. Stakeholders – FedCon employees that work directly with customers are the stakeholders that need to be considered since the improvement of existing technology products and services are highly dependent upon customer feedback and conveying of the feedback to FedCon.
  2. Domain – In terms of innovation, FedCon is interested in creating technology products and services that meet customer expectations and exceed what the competition can offer.

EA is a disciplined approach to accomplishing enterprise objectives through alignment between business and IT. This disciplined approach can also be leveraged to make FedCon more competitive, which can result in bringing technology products and services quicker to the marketplace. This high-level innovation criterion also points towards using TOGAF since it is constantly being improved upon based on the feedback from technology vendors and solution providers.

The culture at FedCon:

The “father of modern management” Dr. Peter Drucker once said that “Culture eats strategy for breakfast.” Culture can affect the ability of any organization to adopt or resist changes to the organization. While culture is typically considered a fuzzy attribute of an organization but there are tangible things that we can observe to decipher corporate culture which includes (1) corporate values, (2) employee recognition and risk-taking, (3) salaries, commission and hourly rates, (4) location, (5) clothes and (6) domain expertise and product/service subcultures.

At FedCon, the culture is such that change is welcomed as long everyone who is affected by it understands its purpose and there is no disruption to normal business processes. This is a two-pronged issue for the selection of an EA framework since even if the value of EA is understood by senior leadership but it is not understood at the BU, program/project and individual levels then it becomes just another information collection exercise.

The primary cultural concerns in choosing an EA framework are:

  1. Stakeholders – FedCon has a process-driven and metrics-monitoring culture. This is one of the reasons that the Program Management Office (PMO) is an important part of FedCon since it provides a consistent process by which programs/projects can be evaluated. In order to incentivize employees to change their behavior for the organizational transformation, human resources and accounting/finance offices are also stakeholders in EA success.
  2. Domain – Culturally, FedCon is interested in creating an atmosphere that encourages employees to take risks and leverage the massive intellectual property it has developed over the years to stay competitive.

One of the reasons for the success of the PMO within FedCon is its process-driven culture. So for the selection of an EA framework, we have to consider what plays into strengths of FedCon. This high-level cultural criterion leads us to TOGAF that provides a methodological approach for EA within an organization. The EAPMO would make use of lessons learned from the PMO to create a successful EA practice.

Execution at FedCon:

Intention without execution is simply thoughts without results. An organization can have great intentions but if it does not operationalize those intentions then all the strategy discussions and documentation it did just an exercise in futility.

At FedCon, execution has two views. One view is the execution based on winning a government contract to deliver technology products and services. The second view is the execution of the corporate strategy that looks into entering new markets, mergers and acquisitions and creating superior technology products and services.

The primary execution concerns in choosing an EA framework are:

  1. Stakeholders – All employees of FedCon at every level are stakeholders in the successful execution of EA.
  2. Domain – In terms of execution, best practices have to be applied/created for all of FedCon and metrics developed that assess organizational performance.


After assessing the business environment of FedCon to determine an appropriate EA framework, next we have to determine people, processes and technologies needed to standup the EAPMO. These needs are discussed below:


In order to assess the skill sets needed to run the EAPMO, we have to look at the current skillsets available, skillsets that people need to be trained on and hiring people with the necessary skillsets at FedCon. The hard skills needed to join the EAPMO require the knowledge of the chosen EA framework (i.e., TOGAF) and the ability to find common themes to enhance collaboration. The soft skills needed to join the EAPMO require (1) being politically aware, (2) ability to create bridges/connections and (3) high emotional intelligence. Additionally, metrics will be created to evaluate EAPMO team members based on their hard and soft skills.


The business processes followed by EAPMO would be determined by TOGAF best practices and what has worked within FedCon. At a high level, this would be the architecture governance process and at the lower level, this would the cross-functional team’s processes for being advocates and collectors of information across FedCon. The various processes would be tested in the first 6 months to work out any wrinkles and get a baseline understanding of what needs to be done.


Now that we have selected the FedCon’s EA framework to be TOGAF, we have to select a tool that supports this framework. This tool can be selected by looking at Gartner’s Magic Quadrant for Enterprise Architecture Tools.


Due to FedCon’s expertise as a technology company and for all the reasons stated in the analysis section, TOGAF is the right EA framework since it provides a roadmap of what needs to be done. One thing to keep in mind is that a framework needs to be flexible enough so it can adapt to changing organizational needs rather than the organization becoming a slave to the framework.


  1. Khan, Arsalan. “5 Factors for Business Transformation.” Arsalan Khan., n.d. Web.
  2. “Stakeholder Management.” ADM Guidelines and Techniques – Stakeholder Management. TOGAF, n.d. Web.
  3. Schekkerman, Jaap. Enterprise Architecture Good Practices Guide: How to Manage the Enterprise Architecture Practice. Victoria, BC: Trafford Pub., 2008. Print.
  4. “Architecture Governance.” Architecture Governance. TOGAF, n.d. Web.
  5. Christensen, Clay M. “Christensen Institute.” Christensen Institute Disruptive Innovation Comments. Christensen Institute, n.d. Web.
  6. “The Business Executive’s Guide to IT Architecture.” The Open Group Architectural Framework (TOGAF) Executive Overview. TOGAF, n.d. Web.
  7. Caldbeck, Ryan. “Why Execution Is Everything In Business.” Forbes. Forbes Magazine, 16 Sept. 2014. Web.
  8. “Organisational Culture Eats Strategy for Breakfast and Dinner.” ORGANISATIONAL CULTURE EATS STRATEGY FOR BREAKFAST, LUNCH AND DINNER. Meliorate, n.d. Web.
  9. Lapkin and Weiss. “Ten Criteria for Selecting an Enterprise Architecture Framework”. Gartner report G00163673. Gartner
  10. Brand, Saul. “Magic Quadrant for Enterprise Architecture Tools.” Gartner report G00263193

Success! You're on the list.
%d bloggers like this: