In 2014, the Digital Accountability and Transparency Act of 2014, or DATA Act, became law. The goal of the DATA Act was to provide more accessible, searchable, and reliable spending data for the purposes of promoting transparency, facilitating better decision-making, and improving operational efficiency. Tracking more than $4 trillion in annual federal spending on a quarterly basis in a clear and consistent way that the public will understand from the entire federal government is a large task. The scale of the effort is overwhelming even with the most modern technology – but the federal data that the DATA Act required Treasury to collect was scattered across hundreds of disconnected systems across the federal enterprise. The new law only gave Treasury three years to collect this data from more than 100 Federal agencies and display all this information for the public. Treasury needed a new approach to accomplish so much work in such a short timeframe. In 2014, agile development, user-centered design and open source code were relatively new concepts that were just getting traction in government. Treasury was an early adopter of these principles to guide the DATA Act Implementation.

B. The Introduction: Background and context

Most technology projects in government follow a “waterfall” methodology for software development – there is a long requirements gathering phase, then the software is developed, is it tested and launched. In contrast, agile development emphasizes working software that does the absolute minimum to achieve a mission, so that user feedback on the initial prototype can be incorporated early and often. Historically Treasury has used a “waterfall” approach to develop government-wide systems and a recent project in similar scope to the DATA Act implementation, took more than 4 years to develop and launch.

In contrast, the DATA Act core data collection instrument, the Broker, took 6 months to develop and deploy. That is a remarkable reduction in the time needed to develop a complex system with more than 100 validation rules and 400 data elements to collect data on $4 trillion in federal spend. This project demonstrates the power of the agile approach. In addition, Treasury’s project engaged hundreds of staff from about 100 federal agencies and significantly expanded the adoption of agile practices across the federal government today.

The Agile development methodology emphasizes collaboration between cross-functional teams and end users, focusing on early delivery of features, adaptive development, and continuous improvement. Agile helps teams rapidly deliver value with a lightweight framework of defining requirements, building to meet them, releasing software, and then repeating the process based on what one has learned along the way. This allows the development team to save time and money by getting ideas in front of users and iterating based on user needs, rather than spending time and money on software that does not fit a use case.

In the Scrum agile approach that the Treasury team took for implementation of the DATA Act, goals were continually set and reassessed for what would be delivered at the end of any sprint, always based on feedback from individual use cases. Additional requests for work to be completed were added to a backlog – a product roadmap of requirements for the entirety of the project – and prioritized by the Product Owner for future sprints. Sprints always included planning for prioritization of key tasks. The amount of time sprints were given usually depended on the planned work’s complexity.

To implement the DATA Act, Treasury teams typically worked in two week sprints, though these could be made longer or shorter depending on the work and hours of the team.

The challenge (or problem that was addressed): Why and what does it matter?

The challenge of tracking the nearly $4 trillion in annual spending on a quarterly basis by the federal government required the collection of more than 400 disparate financial data elements across 100 federal agencies. The approach to meet the challenge required a rethink of the way government works with software development. Treasury took a data-centric approach that avoided large-scale system changes and instead focused on managing the financial data already in existence at more than 100 federal agencies. By using modern software development practices like testing and code review, and working with real users along the way, Treasury busted many ‘can’t do this in government’ myths as it worked to implement the DATA Act in an agile manner.

Analysis : Framing the response (strategy): What approach was chosen to address the goal and why?

The DATA Act implementation involved three key elements to enable effort – the development of consistent government-wide data standards, the development of the data collection and validation tool and the website to display the data for external stakeholders. Treasury used the agile methodology in each of these critical efforts.

Government-wide Standards: DATA Act Information Model Schema (DAIMS)

While the DATA Act outlined the general requirements for federal spending transparency, more detailed data definition standards were needed to ensure consistency and interoperability among the various award and financial communities that share these data elements. In developing those data standards, the Treasury collaborated with the federal community and external stakeholders to create finalized definitions and data standards for agreed upon core financial data elements.

To account for the nearly 400 disparate financial data elements that the DATA Act sought to unite, Treasury created what it termed DAIMS – the DATA Act Information Model Schema. The goal behind DAIMS was to establish standards, or rules that would establish how the data elements are described, represented, and recorded in a consistent format. Standardizing the data makes it more usable to more than just the project or person that created the data. It enables and encourages re-use of the data for other purposes and provides for greater data integrity, accuracy and consistency; clarifies ambiguous meanings; minimizes redundant data; and documents business rules.

Treasury issued the DAIMS v. 1.0 in April 2016. The DAIMS is the most critical component of the DATA Act implementation because it sets the requirements for all the data elements to be reported by federal agencies, the relationships between each element, the validation rules and the overall context of how the data fits together. There are more than 400 interconnected data elements within the DAIMS – some being submitted directly from the agency financial systems and others being derived or pulled from existing government-wide systems.

Finalizing the requirements of the DAIMS was an extensive and labor-intensive process. Treasury worked tirelessly for about a year to develop this model and ensure that the requirements would improve the transparency of federal spending. Treasury issued the baseline version of the DAIMS in May of 2015 and issued five subsequent versions based on ongoing feedback from external stakeholders and federal agencies until April 2016 and during that time we received hundreds of comments that were all addressed by the Treasury team. Treasury addressed longstanding data anomalies and consistency issues by engaging all federal agencies. Treasury received over 600 comments on its draft DAIMS, held over 30 meetings with 20 individual agencies on the DATA Act Schema; completed 2 DATA Act Implementation Playbooks; held over 55 office hours calls with an average of 80 callers per week; and held 17 workshops with agencies and shared service providers.

The DAIMS, as it was developed, is the only authoritative data standard for reporting across multiple U.S. government functions that include budget, accounting, procurement, grants, loans, and insurance. In addition to establishing a repository of definitions for more than 400 common data elements, DAIMS enables effective sharing of spending information; reduces the amount of manual intervention in information processing; provides a means for publishing those definitions for the benefit of information exchange partners; and provides a foundation for linking financial data to other data sets to improve outcomes and identify savings. DAIMS evolved to support US federal spending data requirements, as prescribed by the DATA Act, through schema versioning and release management. DAIMS versioning included major, minor, and maintenance releases.

Standardizing the data makes it more usable to more than just the project or person that created the data. It enables and encourages re-use of the data for other purposes and provides for greater data integrity, accuracy and consistency; clarifies ambiguous meanings; minimizes redundant data; and documents business rules. The team set out to do what some in the federal government said was impossible and we believe the final product is a model that can serve as a foundation for federal data management in the future and help improve the way the government works.

Data Collection and Validation: DATA Act Broker

The DATA Act Broker is a tool that Treasury developed to allow federal agencies to submit the required data in a standardized format. The Broker accepts data submitted directly from agencies and it also pulls data from existing data sources when needed. The Broker validates agency data, allows agencies to certify the data and complete the data submission based on the DAIMS. The Broker then uploads the agency certified data to be posted on the USAspending.gov website.

Treasury has also taken an agile development approach for the Broker and created the initial prototype Broker in 2015. This prototype was tested by agencies and used to develop the Alpha Broker that was released in April 2016, Beta Broker that was released in June 2016 and the full Broker that was completed in September of 2016. During the development there were 220 Broker testing accounts and agencies were heavily engaged in the testing at each stage of development. The Broker was developed using open source code, which can be accessed on the Federal Spending Transparency Collaboration site. By using open source code for the Broker, the government invests in the development of the code once and then agencies can reuse the same code or extend it within their own environments to improve the quality of the data before they submit it to Treasury. Financial management system vendors can also leverage the same code to enhance the agency systems.

It is also important to note that the Broker is a robust, government-wide tool that will improve data quality and contains about 100 complex validations and more than 170 data element validations. In the past, with a waterfall development approach, it would take several years to develop a functional system like the Broker. We developed the Broker in less than 6 months – that is the impact of the agile development process. It allowed us to shave- off a significant amount of development time and resources.

Data Transparency: USAspending.gov

In May 2017, every agency in the federal executive branch began reporting its spending data using the standardized data structure that Treasury and OMB had worked to establish since passage of the DATA Act. That spending data was initially published on a beta site for further refinement – beta.usaspending.gov. Since then, on a quarterly basis, Treasury continues to report data on USAspending.gov that includes obligations and expenditures related to all federal spending by appropriations account, object class, and program activity.

As the repository for all agency spending data, USAspending.gov brings together data that was once scattered across hundreds of databases and paper reports across government and has digitized this information so it can be searched and clearly viewed by the public in one central location. The USAspending.gov website allows taxpayers to examine more than $4 trillion in federal spending each year and see how this money flows from Congressional appropriations to local communities and businesses. Users can search by location (state, city, county), by federal agency or by keyword for a specific policy or type of spending. Users can also search for a specific recipient of federal funds – a business, a university or other entity that you would like to know more about. In addition, the public can link awards with the “federal account” that supplied the funding.
The site was developed using user-centered design principles, which involved asking citizens want they wanted communicated about how the U.S. federal government appropriates and spends taxpayer monies. It was important for the Treasury team developing the USAspending.gov to engage the user community.

Getting down to work: By whom and how was the project carried out?

Treasury looked to the private sector and consulted technology experts in government to influence the new agile implementation approach. Treasury developed a strong coalition of federal agencies to be part of the implementation team that were heavily involved with the planning and testing. Treasury identified experts from other federal agencies to join Treasury’s implementation team full-time to ensure they had the expertise needed in house to lead the development team mostly consisting of external contractors on this start-up initiative. Treasury leveraged most of the concepts from the U.S. Digital Services Playbook to guide the implementation of the DATA Act and lead the first government-wide agile project with about 100 agencies involved.

Treasury also developed a broad coalition of external stakeholders – consisting of state and local government officials, academia, transparency and data advocates, and citizens – to review and be part of the agile development process. Treasury created a DATA Act Collaboration Space on Github– one of the first of it’s kind in government – to provide opportunities for stakeholders to comment on all aspects of the project. Valuable feedback was collected on the development of the initial data standards and the display and functionality of the website. One of the most innovative components for government was the fact that Treasury’s agile sprint backlog is accessible to the public and still is today.[ii] If the public made a request for a feature or feedback on the testing – it could be tracked transparently and the public could see the details and when the feature would be completed. Beyond the online engagement and transparency for all aspects of the project – Treasury also held monthly phone calls with all interested stakeholders with the senior-most Treasury officials to ensure any public concerns were heard and addressed. In addition, all of Treasury’s software code for the entire project is open source so that the code can be reused and available to auditors.

Results and Conclusions

The results from the first government-wide agile project in government are remarkable. Today, Treasury’s USAspending.gov website brings together data that was once scattered across hundreds of databases and paper reports across government and has digitized this information so it can be searched and clearly viewed by the public in one central location. The USAspending.gov website allows taxpayers to examine more than $4 trillion in federal spending each year and see how this money flows from Congressional appropriations to local communities and businesses. Users can search by location (state, city, county), by federal agency or by keyword for a specific policy or type of spending. Users can also search for a specific recipient of federal funds – a business, a university or other entity that you would like to know more about. In addition, the public can link awards with the “federal account” that supplied the funding.

USAspending.gov can be used to inform decision-making, provide new insights on spending in communities and better understand the impact of federal spending. Federal agencies now have a more complete view into the categories of spending across the agency, allowing CFOs and others to make more informed strategic decisions about allocating funds. Agencies can also explore how to link other administrative and performance data to improve outcomes. The USAspending.gov site data allows agencies, Congress, and the public to track obligations and outlays and the rate they are occurring across the government on a quarterly basis to compare spending on like activities and services.

Data Lab Your Guide to America’s Finances

Treasury did not stop there – they are continuously improving and expanding citizen-driven transparency. Clear visualizations and User-Centered Design principles have driven the development of the USAspending.gov website and Treasury used these same principles to develop a sister site – The Data Lab. This newest project aims to provide new visualizations and insights on Treasury’s data. More than 200 one on one interviews were conducted with a random sample of the American public to gauge a sense of taxpayer interest in federal finance, where taxpayers currently get information about government spending, and where Treasury can fill the gaps. What the public wanted was largely visualizations around the trillions in dollars the government spends and takes in as revenues, with more drilled down information into those categories like taxes and spending on grants and contracts. As a result, Treasury placed emphasis on infographics and visualizations to better show the power of the data available on USAspending.gov. The team at the Data Lab want to make federal spending data more accessible, in terms of not only physical access to the data, but also to make it easier for people to understand.

Treasury also recently launched Your Guide to America’s Finances to respond to citizen feedback. Released on Tax Day, April 15, 2019, the digital guide helps raise public awareness of the federal government’s finances by translating the complicated aspects of federal finance into a simple, straightforward, and interactive product.

Posted On: April 30, 2020