Module · Data & Questions

The data each analysis needs, and the questions we construct to get it

HSG's analysis treats the disparity question the way the law requires it to be treated: as something to be tested, not assumed. Every technique below starts from a measured gap between an ownership group and the non-disadvantaged comparison group, then works to separate the part the City can legitimately price on the merits (firm size, age, revenue, bonding, prior experience, industry mix, how often a firm bids and how competitively) from the part that legitimate capability does not explain. Where an unexplained residual survives, the social-capital channels (information access, referral and vouching, mentorship, incumbency and repeat business, prime-sub ties, and credit and bonding relationships) are tested as candidate mediators, and verified business-owner accounts are matched cell-by-cell to corroborate what the numbers show. The City's own 2021 study found an entrenched informal network and repeat use of the same firms in its anecdotal record, which makes these channels the natural first hypotheses to test against five new years of data. No technique presumes a verdict. Each one is built to answer whether a disparity traces to discrimination or to neutral factors, and to report exactly where that answer is robust versus where it depends on a defensible modeling choice. Frederick adds a second layer no ratio book carries: the November and December 2021 policy package (the Small Business Reserve, the MWBE program, and annual aspirational goals set at the 2021 study's measured availability) is a dateable intervention, so the update can be built as an evaluation of a reform era, with the interrupted time-series discipline that word implies. The five-fiscal-year data window will straddle the City's January 1, 2026 migration from IonWave to OpenGov, and the instruments below plan for that seam explicitly. The matrix pairs each technique with the analytical question it answers, the data it needs and why, and the four kinds of questions the study will build: what to ask the City to hand over, what to ask firms in the availability survey, what to ask owners in attributed interviews, and the analytical question the technique itself resolves. We publish the instruments in advance so the City, its vendors, and any later reviewer can see that no question embeds its answer. All sampling and data handling are written for City approval, to respect confidentiality, and to survive reproduction under the Maryland Public Information Act.

Question-design and defensibility safeguards

These rules are what let the questions survive scrutiny. They govern every survey item and interview prompt below, so the instrument never embeds the conclusion it is meant to test.

No presupposition. Every conditional question is gated by a screen first, and the conclusion is never embedded in the stem.
Neutral, behavior-anchored response options, never a discrimination-primed option; closed items are paired with an open follow-up so the respondent supplies the attribution.
Mediation is reported as descriptive coefficient-movement accounting, not causal claim. A residual is described as differential returns and social-capital deprivation, never as proof of discrimination.
Pre-registered before any results are seen: the comparison-group definition, the Benjamini-Hochberg false-discovery-rate correction across the significance grid, the selection-model exclusion restriction, and the ownership coding bands.
Contested controls such as incumbency are reported both ways: with the control is the conservative lower bound, without it is the upper bound that treats the variable as itself a channel.
Ownership classification is reported under three codings as a sensitivity band: self-certified, survey-verified, and registry-verified. State MBE certification and federal DBE certification are kept strictly separate throughout, because their legal bases have diverged.
Subcontract and team analyses are contingent on City data; the 2021 study found zero tracked subcontractors in FY2018, so the network layer leads with the well-populated co-bidding graph, with the prime-sub graph a documented supplement and its coverage disclosed.
Every survey is delivered with a defensibility package: response rates by stratum, nonresponse analysis, design weights to the frame, and documented frame sourcing.
AI assists, it does not decide. The AI-assisted procurement-rules review and any AI-assisted coding surface candidate flags on a pre-registered rubric; a named investigator adjudicates every flag, counsel owns every legal conclusion, and a responsible-AI advisor governs the bias-testing and documentation of each AI step.

Technique by technique

Each method, the data it needs and why, and the questions it drives

Every technique starts from a measured gap and works to separate what legitimate firm capability explains from what it does not. None presumes a verdict.

Oaxaca-Blinder decomposition of the group gap in bid-win rate and award/subcontract dollars

Of the measured gap between an ownership group and the non-disadvantaged comparison group in bid-win rate and in award/subcontract dollars, how much is explained by differences in observable, legitimately-priced firm capability (size, age, revenue, bonding, prior City experience, industry mix, bid frequency, bid competitiveness) versus an unexplained residual, reported per group and per category without presuming the residual is discrimination?

Data needed, and why

Firm-by-solicitation bid/quote outcome panel with win/loss flag across five fiscal years, so a win rate and its denominator exist (bid tabulations, quote logs, and plan-holder records from both the IonWave era and the OpenGov era, joined across the January 1, 2026 migration)
Award and payment dollars per firm per fiscal year by category and method, from the City financial-system accounts-payable extract (the dollars-paid utilization base and dollar outcome)
Subcontract-dollar records linking prime to sub (weak/at-risk: FY2018 had zero tracked subcontractors; City compliance records plus prime payment confirmations, with a coverage statement)
Ownership-group classification per firm with a self-cert / survey-verified / registry-verified coding band (City vendor classification records, City SBE self-certification, MDOT certification directory, SBA records, SAM.gov, calibrated by survey; MBE and DBE flags kept separate)
Firm capability covariates for the explained block: firm age, employment size, receipts, primary and secondary NAICS or NIGP lines, market-area location (registration, Census ABS/CBP, commercial firm data, survey)
Bonding capacity per firm so it sits in the explained block, not the residual (SBA Surety Bond Guarantee, prequalification fields, survey)
Prior City experience / incumbency count per firm, run both with and without it as a contested control (derived from the award spine)
Bid frequency per firm and bid-competitiveness measures where recorded (derived panel; bid-tab price columns, non-responsive flags)
Procurement method and department per solicitation for category-level reporting (bid tabs and payment spine)

Questions to construct

DATA-REQUESTFor each of the past five fiscal years, please provide the complete bid and quote history for City solicitations, including for every participating firm the solicitation number, category, procurement method, soliciting department, firm identifier, submitted bid amount where recorded, responsiveness determination, and whether the firm was awarded, from both the IonWave and OpenGov systems.
DATA-REQUESTPlease provide prime award and payment records and, separately, any subcontractor payment or compliance records for the past five fiscal years at the firm and fiscal-year level, including paid dollars, category, method, and award date, with a documented note where subcontractor data are incomplete.
DATA-REQUESTPlease provide vendor registration and classification records with each firm's reported ownership demographics, years in business, employment size, annual receipts, primary commodity codes, and location, noting which fields are self-reported versus independently verified.
SURVEYIn what year did your firm begin operating, and how many years has it been bidding on government contracts in this region?
SURVEYWhat is the largest single contract value your firm is currently able to bond, and through what mechanism (surety bond line, SBA Surety Bond Guarantee, or none)?
SURVEYIn the past five years, how many times did your firm submit a bid or quote on a City of Frederick solicitation, and how many of those resulted in an award?
INTERVIEWThinking about a specific City solicitation your firm pursued, please describe what factors you believe most affected whether your firm was selected or not selected.
INTERVIEWWere there qualifications, experience floors, bonding levels, or other requirements on City solicitations that your firm met, did not meet, or found difficult to meet, and how did that affect your decision to bid?
ANALYTICALWhat share of the bid-win-rate gap and the award-dollar gap for each ownership group, in each category and method, is explained by observable firm capability differences, and what share remains as an unexplained residual, reported both with and without contested controls such as incumbency?

Interrupted time-series attribution of the 2021 reform package (Small Business Reserve, MWBE program, aspirational goals)

The City adopted its Small Business Reserve, its MWBE program, and annual aspirational goals by resolution in November and December 2021, a dateable intervention. Did the level or trend of each group's participation change after that date, in which categories and procurement bands, and can the change be attributed to the specific program mechanism (the SBR band, contract-specific construction goals, certification growth) rather than to composition shifts or market-wide trends?

Data needed, and why

A monthly or quarterly participation series by group and category spanning the five most recent fiscal years, plus bridge records back toward FY2019 where they exist, so pre- and post-intervention segments both have support (City payment and award records; the 2021 study's FY2014-FY2018 record anchors the deeper baseline)
The adoption record for the reform package: the resolutions amending the Purchasing Policies, effective dates, and the dates each mechanism actually began operating, so the intervention is dated precisely rather than assumed
SBR designation records per solicitation: which solicitations were reserved, in which band, and their outcomes, so the SBR's own lane can be separated from open competition
Contract-specific MWBE goal records on construction above $250,000: the goal set, attainment, and any waiver or good-faith determination
SBE certification roster with certification dates since January 2022, so certification growth (154 firms certified as of the first annual review) can be separated from participation growth
The exemption log (60 exemptions in 2022) with dates, so spend routed around competition is not misread as program effect
A comparison series that the reform could not have touched (for example, categories or dollar bands outside every program mechanism) to absorb market-wide trends
A documented seam plan for the IonWave-to-OpenGov migration on January 1, 2026, so a system change is never misread as a participation change

Questions to construct

DATA-REQUESTPlease provide the resolutions and policy documents that created the Small Business Reserve and MWBE programs, with adoption and effective dates, and any records showing when each mechanism first operated on a live solicitation.
DATA-REQUESTPlease provide, for every solicitation since the programs began, the SBR designation status and band, any contract-specific MWBE goal set, the attainment reported, and any waiver or good-faith-effort determination, with dates.
DATA-REQUESTPlease provide payment and award records for the fiscal years before the programs began, back to FY2019 where records exist, so the pre-reform period can be measured on the same basis as the post-reform period.
SURVEYHas your firm become certified as a small business with the City of Frederick, with SBA, or with MDOT since 2021? If so, in what year, and what prompted the certification?
SURVEYSince 2022, has your firm bid on a City solicitation that was reserved for certified small businesses, and what was the outcome?
INTERVIEWIf your firm has worked with the City both before and after 2021, describe what, if anything, changed in how you learn about opportunities, qualify, bid, or get paid.
INTERVIEWWhat difference, if any, has City small-business certification made for your firm in practice, and what did the certification process require of you?
ANALYTICALFor each group and category, did participation shift in level or trend after the November-December 2021 adoption date, is the shift statistically distinguishable from the pre-period trend and from the untouched comparison series, and does its timing and location match the specific mechanism (SBR band, construction goals, certification) that could have produced it?

Capacity-controlled but-for regressions: win-probability probit/logit P(win|bid) and award-dollar Tobit / Cragg two-part model

Holding a firm's legitimate, race-neutral capacity constant (size, age, revenue, bonding, prior City experience, industry, bid frequency), does owner group still independently change its probability of winning once it has bid, and the dollars it is awarded? Between two otherwise-identical firms that both bid, does owner group move the outcome, or do measured capability differences fully explain the gap?

Data needed, and why

Bid-level outcome records linking each bid/offer/quote to its solicitation, firm, and a win/loss flag across five fiscal years (City financial-system awards joined to bid tabs and quote logs from both procurement platforms)
Award and obligated/paid dollar amount per winning bid with contract and modification history, for the censored Tobit/two-part outcome (City financial system)
Verified owner-group classification per firm under a self-cert / survey-verified / registry-verified band (City records, MDOT directory, SBA, survey, SAM.gov)
Race-neutral capacity controls: years in business, employment size, receipts, bonding capacity, prior City experience count and dollars, certifications (Census ABS/CBP/PUMS, survey, SBA Surety Bond Guarantee, City records)
Bidding-frequency / activity measure per firm, both a control and the exposure variable (bid tabs cross-checked to vendor registration)
Procurement method, category, and department per solicitation as fixed-effect / stratification controls (City financial system; solicitation records)
Subcontract / prime-sub award and payment records for the subcontract-dollar version (weak/at-risk; degrades to prime-only with the gap disclosed)
A City-approved sampling and group-aggregation/power-analysis plan for thin cells (HSG methodology submitted for City approval)

Questions to construct

DATA-REQUESTPlease provide, for each of the last five fiscal years, a record of every solicitation, naming the solicitation number, procurement method, category, and awarding department.
DATA-REQUESTPlease provide, for each solicitation, the list of every firm that submitted a bid, quote, or offer, with a flag indicating whether that firm was the awardee.
DATA-REQUESTPlease provide the award amount and all obligated and paid dollar amounts per awarded contract, including modifications, for each of the last five fiscal years from the City financial system, with a formal award identifier where one exists and the source fields needed to construct one where it does not.
DATA-REQUESTPlease provide subcontractor award and payment records and prime payment confirmations where they exist, with the prime, the subcontractor, the dollar amount, and the parent solicitation.
SURVEYIn how many years has this firm been in business under current ownership?
SURVEYWhat was this firm's approximate gross revenue and full-time-equivalent employee count in the most recent year?
SURVEYWhat is the largest single contract value this firm is currently bonded to perform, and what is its aggregate bonding capacity, if any?
SURVEYDuring the last five years, approximately how many City of Frederick solicitations did this firm submit a bid, quote, or offer on?
ANALYTICALHolding firm capacity constant, does owner group have an independent and statistically significant effect on the probability of winning conditional on having bid, and on the dollars awarded, reported per group-by-category-by-outcome cell with and without contested controls such as incumbency?

Heckman two-stage selection correction on the decision to bid, then outcome conditional on bidding

Are the firms that choose to bid systematically different from the available firms that do not, and once that selection is corrected, does owner group still affect outcomes? Is part of the measured disparity occurring upstream at the decision to bid at all, driven by neutral barriers such as awareness, bonding, and set-up cost versus discouragement, rather than at the win stage where most studies look?

Data needed, and why

A roster of available ready-willing-able market-area firms that did NOT bid, not just bidders, to build the first-stage P(bid) equation (City-approved stratified availability survey, de-duplicated commercial firm frame, vendor registration origins)
At least one valid exclusion restriction that shifts bidding but not win merit (distance to Frederick, prior awareness, registration status, bid-preparation fixed cost), from the survey, registration timestamps, and geocoded location
Firm-level reasons for bidding or not, coded as neutral barrier versus discouragement (survey closed-set items corroborated by verified interviews)
The same capacity block and owner-group classification as the win/dollar models, in both stages
City solicitation-awareness and outreach footprint per category (the monthly procurement forecasting page, vendor notification lists, and outreach records, corroborated by survey)
City standing eligibility requirements actually applied per category: bonding, insurance, experience floors, prequalification (purchasing policies, solicitation specifications)
Plan-holder / document-download records from IonWave and OpenGov, the cleanest marker of aware-but-selected-out firms (weak/at-risk across the migration; coverage disclosed)

Questions to construct

DATA-REQUESTPlease provide, for each solicitation in the last five fiscal years, the plan-holder or document-download list of firms that obtained the solicitation documents, distinguished from the firms that ultimately submitted, from both the IonWave and OpenGov systems.
DATA-REQUESTPlease provide the vendor-notification and outreach records showing where each category of solicitation was advertised and which registered vendors were notified, including distribution of the City's monthly procurement forecast.
DATA-REQUESTPlease provide the standing eligibility requirements applied to each procurement category, including bonding thresholds, insurance minimums, prior-experience floors, and prequalification requirements.
SURVEYBefore being contacted for this study, how aware was this firm of City of Frederick contracting opportunities in its line of work, on a scale from not at all aware to very aware?
SURVEYFor the most recent City opportunity this firm was eligible for, did it bid, and what were the main reasons it did or did not bid? Select all that apply: did not know about it, requirements such as bonding or insurance were out of reach, the contract was too large or bundled, the experience or prequalification requirement could not be met, the cost of preparing a bid was too high, prior experience with the process, expected the outcome would not be fair, chose to pursue other work, other.
SURVEYHow much staff time and out-of-pocket cost does it typically take this firm to prepare a City bid, and has that cost ever caused the firm to skip an opportunity it was qualified for?
INTERVIEWWalk me through a specific recent City opportunity your firm was qualified for but did not pursue. What happened, how did you learn of it or not, and what would have had to be different for you to bid?
ANALYTICALAfter correcting for non-random selection into bidding, does owner group significantly affect outcomes, and how much of the overall disparity arises at the decision-to-bid stage versus the win stage, with the upstream portion attributed to identified neutral barriers versus an unexplained discouragement residual?

Within-firm gap x procurement-method interaction with firm fixed effects

Within the same firm over time, is the group win/dollar gap larger in high-discretion, high-trust lanes (sole-source, small purchase, informal quote, exemption-routed buys) than in open sealed competition? If the gap concentrates in high-discretion lanes after differencing out time-invariant firm quality, that is the informal-network signature the City's own 2021 anecdotal record describes, and it is hard to dismiss as unobserved ability. It does not by itself prove discrimination; it locates where any disparity lives along the procurement-method dimension.

Data needed, and why

Firm-level panel of every bid/quote/award event across five fiscal years keyed to a stable normalized vendor entity ID that survives the IonWave-to-OpenGov migration, so firm fixed effects can absorb time-invariant quality (City financial system joined to bid/quote/award records; entity de-duplication with human verification)
Procurement-method code on every event coded to a pre-registered low/mid/high discretion scale, including the SBR band and exemption-routed purchases (purchasing policies and method fields; coding submitted for City approval)
The exemption log with reason, department, and dollar for each exemption, so the highest-discretion lane is observable rather than inferred
Outcome variables per event: bid-win indicator and award/subcontract dollars (bid tabs plus the City ledger; subcontract dollars where tracked)
Survey-verified ownership flags with self-cert and registry codings as a sensitivity band (City records, MDOT directory, SAM.gov, corrected by survey)
Race-neutral capacity controls that vary within firm over time: firm age at bid, current-year receipts, bonding capacity, prior-experience count, bid frequency, solicitation category, plus contested incumbency both ways (ABS/CBP, survey, SBA Surety Bond Guarantee, City ledger)
Sufficient within-cell sample of firms that bid in both a high- and a low-discretion lane by group and category (derived panel; flagged as a feasibility limitation if thin)

Questions to construct

DATA-REQUESTPlease provide, for the five fiscal years, a record-level extract of every solicitation, bid, quote, and award, each record tagged by procurement method (open competition, informal quote, small purchase, SBR-reserved, sole source, emergency, cooperative or piggyback, exemption), the awarded and paid dollar amount, the win or loss outcome, the soliciting department, and a vendor identifier consistent across all five years and both procurement systems.
DATA-REQUESTPlease provide the City purchasing policies that define each procurement method, including the dollar thresholds and the level of buyer discretion permitted under each, and the complete exemption log with the reason, department, and dollar amount of each exemption.
SURVEYIn the past five years, has your firm submitted bids or quotes to the City of Frederick through more than one type of procurement, for example both formal sealed solicitations and informal quote or small-purchase requests?
SURVEYFor each procurement type your firm has used with the City, how did your firm first learn that the opportunity was available?
INTERVIEWThink of a specific City opportunity that was awarded without a full open competition, such as a small purchase, a quote request, or a sole-source award. Walk me through how you became aware of it and how you were or were not invited to compete.
INTERVIEWWhen the City selects a vendor through an informal or discretionary process rather than open bidding, what would have made it easier for your firm to be considered?
ANALYTICALHolding the same firm fixed and controlling for time-varying capacity, does the estimated group difference in win probability and award dollars differ across low-, mid-, and high-discretion procurement methods, including the exemption lane, and is any such difference statistically significant per group and category?

Dynamic state-dependence: effect of a first City award on probability of a second, by group

After differencing out fixed firm quality, does winning a first City award raise a firm's probability of winning a second, and is that incumbency or learning premium smaller for minority- and women-owned firms than for comparable non-MWBE firms? A smaller second-award boost for a group is consistent with the repeat-use-of-the-same-firms pattern the 2021 study documented. The test is corroborative, not dispositive: awards build network capital and network capital wins awards, so reverse causation is acknowledged and the finding is labeled differential returns and social-capital deprivation, never proof of discrimination.

Data needed, and why

Firm-level panel with the full date-ordered City award history per firm across five fiscal years, so first and subsequent awards can be sequenced (City financial system ordered by award/payment date on the normalized vendor key)
An initial-conditions baseline: each firm's pre-period vendor status, registration date, and prior experience at window start, to separate true state dependence from unobserved heterogeneity (registration plus earliest historical award extract; weak/at-risk)
First-time-vendor and lost-contract / non-responsive records to define the correct at-risk set for a second award (bid tabs, non-responsive logs)
Survey-verified ownership flags by firm with self-cert/registry band, since the test is by group (City records, MDOT directory, SAM.gov, corrected by survey)
Time-varying capacity controls at the time of the first award plus the dollar size and method of that first award (City ledger; ABS/CBP, SBA Surety Bond Guarantee, survey)
Subcontract / prime-sub history per firm as a corroborating network-capital measure (compliance records; weak/at-risk, reported under caveat)

Questions to construct

DATA-REQUESTPlease provide a date-ordered award and payment history for every vendor paid by the City during the five fiscal years, including each award date, awarding department, dollar amount, procurement method, and a vendor identifier consistent across years and systems, plus award history predating the window where available so each vendor's first City award can be identified.
DATA-REQUESTPlease provide the records of firms that submitted a bid or quote but did not receive an award, including non-responsive determinations, for the five fiscal years.
SURVEYAfter your firm received its first contract or purchase order from the City of Frederick, did the chance of winning additional City work change, and if so, in what way?
SURVEYFollowing your firm's first City award, did any of the following change: invitations to bid on later opportunities, introductions to prime contractors or other departments, your bonding capacity, or your access to past-performance references?
INTERVIEWTell me about the period right after your firm won its first contract with the City. What happened next in terms of being considered for additional work, and what helped or got in the way of winning a second contract?
INTERVIEWLooking back, do you attribute the path from your first City contract to later contracts more to your firm's performance and capacity, to relationships and referrals, or to something else?
ANALYTICALControlling for firm capacity and first-award characteristics and accounting for initial conditions, does receiving a first City award raise the probability of a second, and is that marginal effect significantly smaller for minority-owned or women-owned firms than for otherwise comparable firms, per group and category?

Social network analysis of prime-sub and co-bidding structure (bipartite plus co-bidding graph, ownership homophily/assortativity, brokerage/centrality, MRQAP dyadic regression)

When teaming, subcontracting, and co-bidding relationships are mapped as a network, do firms connect along ownership lines (minority- and women-owned firms clustering with each other and away from well-connected primes), and does a firm's network position predict who wins City work after capability is controlled? The Fourth Circuit's controlling disparity case turned in part on evidence of an informal, racially exclusive network, and the City's own 2021 study described an entrenched informal network in its anecdotal record. This technique tests whether that structure is present in the transaction data itself, or whether relationships sort on neutral factors such as industry overlap, size, geography, and bonding capacity.

Data needed, and why

Prime-to-subcontractor payment and award records linking each prime contract to subs actually used and dollars paid (firm identity, ownership flags, industry, dollars, dates), the raw material for the bipartite graph (City compliance records and required utilization reports; weak/at-risk: FY2018 had zero tracked subcontractors, so coverage must be audited first)
Bid tabulations and plan-holder/quote logs across five fiscal years listing every bidder, not only winners, to reconstruct the co-bidding graph as the lead layer (City bid tabs, quote logs, plan-holder lists from both platforms)
Team and partnering declarations submitted with bids, including for proposals that did not win (procurement files, SAM.gov entity relationships)
Firm attribute table per node: ownership group, industry, size, location, years in business, bonding capacity, registration status, used as MRQAP covariates (City registration and certification records, Census ABS/CBP, commercial firm data, SBA Surety Bond Guarantee, survey)
Survey-reconstructed teaming and referral edges (firm-reported primes subbed to, firms teamed with, how work was learned of), the documented workaround for the weak subcontract layer (availability survey)
A record-completeness audit of subcontract/team coverage by year and department, to bound and disclose network-analysis coverage (City system metadata, data dictionary, extract logs)

Questions to construct

DATA-REQUESTFor each City contract awarded during the study period, please provide the subcontractors used and dollars paid to each, including name, address, ownership classification if recorded, industry code, and payment dates, and identify the system of record and the date range for which these records are reliably populated.
DATA-REQUESTPlease provide complete bid tabulations and plan-holder or quote logs for every solicitation in the study period, listing all firms that submitted, not only the awardee, so firms appearing together in the same competitions can be identified.
DATA-REQUESTWhere bids included team or partnering arrangements, please provide the submitted documents that name the team members and their roles, including for proposals that were not selected, and a coverage statement by fiscal year and department for the share of contract actions and dollars for which subcontractor and team information is present.
SURVEYIn the past five years, which firms has your business worked with as a subcontractor or team member on a contract, public or private, and which firms has your business engaged as your subcontractors or team members?
SURVEYHas your business ever been invited by a prime contractor to join a team or provide a subcontract bid on City work, and approximately how many such invitations have you received in the past five years?
INTERVIEWIf you have sought to team with or subcontract under a prime contractor on public work and were unable to form that relationship, what reasons were given or what obstacles did you encounter?
INTERVIEWPlease describe how the teaming relationships you do have came about, for example through prior coworkers, trade associations, family or community ties, repeat business with the same firms, or formal matchmaking programs.
ANALYTICALAfter accounting for industry overlap, firm size, geographic proximity, and bonding capacity, do prime-sub and co-bidding ties form disproportionately between firms of the same ownership group, and does network position predict City award outcomes, or are the observed ties and outcomes explained by those neutral factors?

With-and-without-contested-controls reporting on the per-(group x category x outcome) significance grid

For each combination of ownership group, procurement category, and outcome (bid-win rate and award/subcontract dollars), is there a statistically significant disparity, and how sensitive is that finding to whether the model includes controls that may themselves be products of past exclusion, namely incumbency and prior City experience? Does a disparity hold up when everything including the contested variables is controlled (the conservative read), and does it grow when those variables are excluded (the read that treats them as channels of disadvantage), so the City can see exactly where the conclusion is robust versus modeling-dependent?

Data needed, and why

Firm-level outcome records by group, category, and outcome type: bids submitted, bids won, dollars awarded, and subcontract dollars where available, across five fiscal years (City financial system, bid tabs, subcontractor records)
Incumbency indicator with a clean date sequence: whether the firm had a prior City award before the action and how recently, entered both ways (derived from award history)
Prior-experience measures: count and dollar value of prior relevant contracts (City and, where available, other public or private) and years in business, entered both ways (award history, SAM.gov past performance, registration, survey for non-City experience)
Uncontested capacity controls held constant in both specifications: firm size, bonding capacity, industry, and the category/method of each action, so only the contested variable changes (registration, Census ABS/CBP, SBA Surety Bond Guarantee, survey, method coding)
A consistent group and category taxonomy aligned to the 2021 study's five categories (Construction, Architecture & Engineering, Professional Services, Other Services, Goods) so the update reads against the baseline grid, with cell-size counts, a documented minimum-cell-size rule, and the pre-registered false-discovery-rate correction
Survey-verified ownership flags to validate the administrative classification that defines every grid row (availability survey)

Questions to construct

DATA-REQUESTPlease provide, for each firm and each contract action in the study period, the award and bid history needed to determine whether the firm was an incumbent or had prior City experience at the time, including award dates and dollar amounts, so prior-relationship status can be constructed in date order.
DATA-REQUESTPlease provide the procurement method and category for each solicitation and award (for example open bid, small purchase, SBR-reserved, sole source, cooperative contract) so outcomes can be tabulated by category on the same five-category basis the City's 2021 study used.
DATA-REQUESTPlease provide your standard ownership and certification classification for each vendor, along with how and when that classification was recorded, so it can be reconciled with firm self-reported ownership.
SURVEYApproximately how many relevant contracts has your business performed in the past five years for public and for private clients, and how many years has your business been operating?
INTERVIEWPlease describe how prior experience or being an existing City vendor has affected your ability to win additional City work, in either direction, and how your business first obtained its first public contract.
ANALYTICALFor each combination of ownership group, procurement category, and outcome, is the measured disparity statistically significant both when incumbency and prior experience are controlled and when they are excluded, and where do the two specifications agree versus diverge?

AI-assisted review of the City's own procurement rules for internal conflicts and disparate impact (human-adjudicated, RAI-governed)

Do City of Frederick procurement policies, manuals, and standing solicitation templates contain rules that conflict with one another, or that though facially neutral mechanically screen out ready-willing-able diverse firms more than the work requires, and can a large-language-model read of the full rules corpus surface those conflicts and candidate disparate-impact provisions faster and more completely than a manual read, for a named investigator and legal counsel to then verify? The City's own materials already show one candidate: the SBR band is described differently on the 2026 program form than in the City's web and press materials, and reconciling the two is a concrete assessment task.

Data needed, and why

The full City procurement policy corpus: the Purchasing Policies as amended in 2021, administrative regulations, standing solicitation templates and boilerplate, prequalification rules, bonding and insurance schedules, delegation-of-authority and method-threshold tables, and the SBR and MWBE program forms and guidance in each version used since 2021
The five-fiscal-year solicitation corpus already requested for the criterion audit, so a flagged rule can be tied to the solicitations that carry it (City solicitation archives across both platforms)
Availability-survey and census firm-attribute distributions (firm age, bonding capacity, receipts by ownership) needed to estimate, for each flagged criterion, the share of each ownership group that could satisfy it (availability survey; Census ABS/CBP)
A pre-registered coding scheme mapping each rule to the barrier framework and a four-fifths disparate-impact screen, so the AI pass produces candidate flags on a fixed rubric rather than open-ended judgments
A logged human-in-the-loop review record: every AI-surfaced conflict or disparate-impact flag is adjudicated by a named investigator, and every legal conclusion by counsel; the AI makes no finding

Questions to construct

DATA-REQUESTPlease provide the complete current and study-period procurement policy corpus for the City, including the Purchasing Policies as amended by the 2021 resolutions, standing solicitation templates and standard terms, prequalification and eligibility rules, bonding and insurance schedules, the tables that set procurement-method thresholds and buyer discretion, and every version of the SBR and MWBE program forms and guidance issued since 2021.
DATA-REQUESTWhere procurement rules changed during the five-year study period, please provide the dated versions so each rule can be attributed to the solicitations issued under it.
SURVEYIn your firm's experience, are there specific City requirements that seem to conflict with one another, or that appear unnecessary for the work, such that they made it harder to register, qualify, or bid? Please describe.
INTERVIEWWalk me through any City requirement that struck you as contradictory, outdated, or out of proportion to the job, and describe how it affected your firm's ability to compete.
ANALYTICALAcross the full procurement-rules corpus, which provisions conflict with one another (beginning with the SBR band as described on the 2026 form versus the City's other published descriptions), and which facially neutral provisions can be satisfied by a significantly smaller share of the available minority- or women-owned pool than of the comparison pool, once each AI-surfaced flag is verified by the investigator and, for legal conclusions, by counsel?

What we ask the City of Frederick to provide

The thirteen-item data request

The study stands or falls on the records the City turns over. We name them explicitly and in advance, sequenced for the IonWave-to-OpenGov migration the five-year window straddles, so nothing is assumed and Phase 2 starts on a verified base.

01Transaction-level payment records for the five most recent fiscal years from the City financial system: every payment with vendor identifier, dollar amount, payment date, department, and commodity coding, spanning both the IonWave era and the OpenGov era with a documented crosswalk across the January 1, 2026 migration, plus bridge-year records back toward FY2019 where they exist for the reform-era baseline.
02Award and contract records for the same five fiscal years: solicitation number, procurement method, awarding department, award date, awarded and not-to-exceed amounts, and modification history, with a formal award identifier where one exists and the source fields needed to construct one where it does not (the 2021 study's first data-reform recommendation), including awards under $50,000.
03The complete bid and quote history per solicitation: every firm that submitted a bid, quote, or offer, submitted amounts where recorded, responsiveness determinations, and an awardee flag, together with plan-holder and document-download logs from both IonWave and OpenGov, so aware-but-did-not-bid firms are visible.
04The commodity-code dictionary and the NIGP (or successor) coding actually applied to each transaction, with a note on the use of general-services codes such as 961 and 962, so category assignment can be audited and corrected rather than inherited (the 2021 study flagged vague commodity coding as a data-architecture gap).
05Prime-to-subcontractor records: subcontractor utilization plans submitted at bid, post-award subcontractor payment confirmations, and good-faith-effort documentation on construction contracts above $250,000 carrying MWBE goals, with a coverage statement by fiscal year and department, since the 2021 study found zero tracked subcontractors in FY2018 and the update must measure, not assume, how far that gap has closed.
06The SBE certification roster and outcomes: every firm certified since January 2022 with application and decision dates and certification path (SBA certification, MDOT, USDOT, or City self-certification), and every SBR-designated solicitation with its band, the certified firms that competed, and the outcome, so the reserve's own lane can be evaluated directly.
07MWBE program records: the annual aspirational goal-setting record for each year, contract-specific goals set on construction above $250,000, reported attainment against each goal, and every waiver or good-faith-effort determination with its justification.
08The exemption log: every exemption from competitive procurement granted during the study period (60 were reported for 2022), with date, department, dollar amount, and stated reason, so spend routed around competition can be analyzed rather than lost.
09Vendor registration and classification records: the vendor file with reported ownership demographics, self-identified small/minority/women-owned flags, years in business, size fields, commodity codes, and location, noting which fields are self-reported versus independently verified, together with any state MBE and federal DBE flags kept as separate fields.
10The purchasing-policy corpus in every version in force during the study period: the Purchasing Policies as amended by the November and December 2021 resolutions, procurement-method definitions with dollar thresholds and discretion levels, standing solicitation templates and standard terms, bonding and insurance schedules, and each version of the SBR and MWBE program forms and guidance.
11The annual program-effectiveness record: the yearly MWBE and SBR effectiveness reviews the 2021 policy amendments require (the 2022 review is public; any FY2023-FY2025 reviews, internal or published), plus DEI Advisory Board materials from its annual procurement reviews.
12Accounts-payable timing records: invoice date, payment date, and payment amount for prime and, where captured, subcontractor payments, so days-to-pay can be tested by firm ownership rather than left as anecdote.
13Business names on building and trade permits issued during the study period, the 2021 study's own recommendation for finding firms that work in the City but never enter its procurement system, plus vendor outreach and notification records showing how and where opportunities were advertised.

Survey

Availability survey item bank

Firm-facing items measuring ready, willing, and able status, capability, bidding behavior, and the social-capital channels, with owner demographics for verification.

1Does your firm currently perform any of the following categories of goods, services, or construction? Select all that apply. [checklist keyed to the City's five procurement categories: Construction / Architecture & Engineering / Professional Services / Other Services / Goods]
2In the past five years, has your firm performed work for any government, institutional, or large private client in these categories? Select the client types that apply. [None / Municipal government / County government / State / Federal / School system or university / Large private firm]
3Would your firm be willing to bid on a City of Frederick contract in these categories under typical solicitation terms? [Yes / No / Depends on the specific opportunity]
4In what year was your firm established, and how many years has it been bidding on government contracts in this region? [numeric]
5Approximately how many full-time-equivalent employees did your firm have at the end of the last fiscal year? [numeric]
6Approximately what were your firm's total gross receipts in the last fiscal year? [banded numeric ranges]
7What is the largest single contract, by dollar value, your firm has completed in the last five years? [banded numeric ranges]
8What is the maximum bonding capacity your firm can currently obtain on a single project, if any, and through what mechanism? [None / Under 250k / 250k to 1M / 1M to 5M / Over 5M / Do not know], [surety bond line / SBA Surety Bond Guarantee / none]
9Has your firm ever been unable to pursue a contract opportunity because it could not obtain a required bond? [Yes / No / Not applicable, bonding not required for our work]
10In the last two years, has your firm sought external financing (a loan, line of credit, or surety credit) for working capital or to perform a contract? [Yes and obtained it / Yes and was denied or got less than requested / No, did not seek financing]
11Does your firm have an established relationship with a bank or lender and with a surety provider that you can draw on for a contract, separate from the dollar amount you are able to bond? [Yes both / Bank only / Surety only / Neither]
12How does your firm typically first learn about City of Frederick contracting opportunities, and at what point in the process? Select all that apply. [OpenGov portal postings / The City's monthly procurement forecast page / City website / Email from a buyer or department / A prime contractor or business contact / An industry association or chamber / A networking event / Word of mouth / We do not currently learn about them]
13How would you rate the timeliness and completeness of the information your firm receives about City opportunities before a bid is due? [More than adequate / Adequate / Inadequate / We do not receive it]
14When your firm pursues new work, how often does an introduction, recommendation, or referral from someone already working with the buyer play a role? [Always / Often / Sometimes / Rarely / Never]
15In the last five years, has a prime contractor invited your firm to join a bid team or subcontract on a City or other public project, and approximately how many such invitations? [Yes more than once / Yes once / No], [numeric]
16When your firm acts as a prime, how do you most often identify subcontractors? Select all that apply. [Firms we have worked with before / Referrals from other contractors / Public directories or certified-firm lists / Open solicitation to subcontractors / We do not use subcontractors]
17Which firms has your business worked with as a subcontractor or team member in the past five years, public or private, and which firms has it engaged as your subcontractors or team members? [open list]
18Has your firm participated in a mentor-protege, technical-assistance, or bid-readiness program offered by any government or organization? [Yes currently / Yes in the past / No never / Was not aware any existed]
19Approximately how many separate City of Frederick contracts or purchase orders has your firm been awarded in the last five years? [None / 1 / 2 to 5 / 6 to 20 / More than 20]
20If your firm has done City work, did your first award lead to additional awards from the City, and what changed afterward (invitations to bid, introductions to primes or departments, bonding capacity, past-performance references)? [Yes / No / Only one / None], [select all that apply]
21In the last five years, how many times has your firm submitted a bid, quote, or proposal to the City of Frederick? [numeric]
22For the most recent City opportunity your firm was eligible for, did it bid, and what were the main reasons it did or did not? Select all that apply. [Did not learn about it in time / Contract too large or bundled / Bonding or insurance requirement out of reach / Experience or prequalification requirement could not be met / Cost of preparing a bid too high / Prior experience with the process / Did not expect a fair chance / Chose to pursue other work / Other]
23How much staff time and out-of-pocket cost does it typically take your firm to prepare a City bid, and has that cost ever caused the firm to skip an opportunity it was qualified for? [numeric / narrative], [Yes / No]
24Through which procurement methods has your firm most often won public work? Select all that apply. [Sealed competitive bid / Request for proposals / Small or micro purchase / Small Business Reserve solicitation / Sole-source or single-source / Cooperative or piggyback contract / Have not won public work]
25Is your firm currently registered or certified under any of the following programs? Select all that apply. This information is recorded in separate fields and never combined across programs. [City of Frederick SBE self-certification / SBA certification (8(a), HUBZone, WOSB, or size determination) / MDOT MBE certification / USDOT DBE certification / Other local or regional certification / None]
26Is your firm at least 51 percent owned and controlled by one or more individuals in any of the following groups? Select all that apply. This information is used only in aggregate for the study. [Woman / Black or African American / Hispanic or Latino / Asian American / Native American or Alaska Native / Native Hawaiian or Pacific Islander / Veteran or service-disabled veteran / None of these / Prefer not to answer]
27Approximately how far is your firm's primary place of business from the City of Frederick, and in what city, county, and state is it located? [numeric / text]
28What is the highest level of formal education completed by the firm's primary owner? [optional; banded categories]

Interview

Attributed interview prompts

Owner-facing prompts that corroborate the statistical channels and distinguish discrimination from neutral factors, gated so they never presume a verdict.

1Tell me about your firm and the kinds of work it does. How did you come to start or run it?
2Walk me through the most recent time your firm pursued a City of Frederick opportunity, from how you first heard about it to how it ended.
3How does your firm usually find out about contracting opportunities, and how far in advance? Has timing ever affected whether you could compete?
4When you have won or nearly won work, what role, if any, did knowing someone, an introduction, or a recommendation play? When you have lost, do you know who won and why?
5Please describe, for a specific opportunity, how you found out about it, who if anyone connected you to the City or to a prime, and what relationships or sources of information you think made the most difference to the outcome.
6Describe your experience working with or being invited by prime contractors on public projects. How are those teaming relationships formed in your experience, and if you sought one and could not form it, what obstacles did you encounter?
7Have you had access to a mentor, an experienced firm, or a program that helped you become bid-ready? What difference did it make, or what was missing?
8When your firm did not have prior City experience, how did that affect your ability to compete, and what happened after your first award, if you had one?
9Tell me about your experience with bonding, insurance, or financing requirements on public work. Were any of these a deciding factor in whether you bid or could perform?
10Have you ever decided not to bid on City work you were qualified for? Walk me through that decision, and what would have had to be different for you to bid.
11If your firm is certified as a small business with the City, with SBA, or with MDOT, walk me through what the certification process required and what the certification has and has not changed for your firm.
12If your firm has worked with the City both before and after 2021, describe what changed, if anything, in how you learn about work, qualify, bid, or get paid.
13Thinking about how contracts are sized or specified, have you encountered requirements that made it hard for a firm like yours to qualify or compete? Which ones, and on what solicitation?
14Has the speed of payment after you completed public work ever affected your firm's cash flow or your willingness to take on more public contracts?
15Think of a specific City opportunity awarded without a full open competition, such as a small purchase, a quote request, or a sole-source award, and a large sealed bid. Comparing across these purchase types, do you notice any difference in how your firm fares? Describe an example.
16In your private-sector or commercial work, have you experienced anything similar to or different from what you have described about public contracting? How do they compare?
17Of everything you have described, which one or two factors have most shaped your firm's ability to win and perform public work? Were they about your firm's capacity, about access and relationships, or about how you were treated?
18Is there anything I did not ask about that would help the City understand the obstacles or advantages your firm has experienced in its contracting?

How this maps to the City's forthcoming RFP (draft under review)

Draft scope Task 1, initiation and data collection across the five most recent fiscal years: the thirteen-item data request above is that task made concrete, sequenced for the IonWave-to-OpenGov seam, and delivered with a data dictionary and coverage audit so Phase 2 analysis starts on a verified base rather than an assumed one.
Draft scope Task 2, the disparity study: the decompositions, but-for models, selection correction, within-method comparisons, and the per-group per-category significance grid produce findings on the same five-category basis the City's 2021 study used, so the update reads against the baseline instead of resetting it.
Draft scope Task 3, the supplier diversity programs assessment: the interrupted time-series technique treats the November-December 2021 policy package as a dateable intervention and evaluates each mechanism (the SBR band, contract-specific construction goals, certification growth, the exemption lane) on its own outcomes, which is what the required yearly effectiveness reviews were built to feed.
Draft scope Task 4, the legal and policy assessment: the procurement-rules review and criterion audit supply the evidence layer, every legal conclusion is owned by counsel, and the task is scoped as defensibility review, not legal advice, matching the draft's own exclusion.
Draft scope Task 5, stakeholder engagement: the availability survey, the attributed interview protocol, and the publish-in-advance posture of this page are the engagement instruments, recruited broadly across Frederick's contracting community so the record reflects firms that win City work, firms that lost, and firms that never entered.
Draft scope Task 6, findings, recommendations, and the strategic implementation plan: the cause-to-action mapping on the approach page converts each isolated cause into a specific City lever, race-neutral first, with any race-conscious element gated on the significance grid, severable, and sunset-dated.
The draft's two-phase structure: the instruments, the data request, and the pre-registered methodology are Phase 1 foundations built to carry into Phase 2 analysis and reporting without rework in between.
Mapping basis: the City's RFP is forthcoming (draft under review) and carries no solicitation number yet. This mapping tracks the draft scope of work and will be re-keyed to the RFP's own section structure when it is issued.

See the method behind the questions

The approach module sets out how we move from a measured gap to its cause. The methodology module places the engine inside the legal framework a Maryland record must satisfy in the Fourth Circuit.

Our approach Methodology & legal framework