Background

In this article, you will find the information presented in the Agency Administrator and Data Quality Training Webinar. The article covers the main responsibilities of the Agency Administrators, important theory of data quality, and practical recommendations on how to monitor data quality at your agency. 

Note: You can see the recording of the webinar here.

 

Topics

Agency Administrator

Who is the Agency Administrator (AA)?

Requirements to be an Agency Administrator? 

Main Responsibilities of the Agency Administrator

Data Quality

Concept of Data Quality

Data Quality Components

• Data Timeliness
• Data Completeness
• Data Accuracy
• Data Consistency

Common Data Quality Errors

Relationship to head of household (HoH) Error

Disabling Condition Error

Date of Birth Error

Data Quality Report Cards

Step 1: 211OC sends DQ Corrections to Agency Administrators

Step 2: Agencies complete DQ Corrections in HMIS

Step 3: 211OC publishes Data Quality Report Cards

Data Quality Report Cards Due Dates

Takeaways

 

Agency Administrator

Who is the Agency Administrator (AA)?

The Agency Administrator is the person, or persons, who oversee and manage the data your agency enters in the OC HMIS. The AA is the main contact between the agency and the HMIS Department. 

 

Requirements to be an Agency Administrator? 

Agency Administrators don't have to be supervisors, managers, directors, or have any special job title. Any HMIS user who meets the following characteristics can be assigned as the AA:

• Is able to provide basic technical support 
• Is reliable and available for the rest of the HMIS users
• Is informed of all the OC HMIS updates and share them with the rest of the HMIS users

 

Main Responsibilities of the Agency Administrator

• Managing Users: the AA is responsible for managing the rest of the HMIS users at their agency. This includes:
  • Identify the users who need access to HMIS
  • Inform the HMIS Help Desk when a new user completes training
  • Provide further training when users need it
  • Notify the Helpdesk when a user no longer needs access to HMIS so that their account can be deactivated
  • IMPORTANT: when there are changes in the Agency Administrator assignation, Executive Directors should submit the contact information of the new Agency Administrator to the HMIS Help Desk 

 

• Communication: the AA is in charge of the communication related to HMIS with the rest of the users. This includes: 
  • Share important HMIS news with all the users at the agency
  • Update user on topics discussed at User and Data meetings
  • Ensure users are getting the HMIS Help Desk emails correctly

 

• Technical Support: the AA has the responsibility of providing technical support to all the users at their agency. This includes:
  • Answer all the technical support question a user might have
  • Escalate the issues that can't be resolved to the HMIS  Help Desk 
  • IMPORTANT: Only Agency Administrators should submit tickets to the HMIS Help Desk
  • Best Practices for entering effective tickets to the Help Desk: 
    • Provide as many details as possible so we can recreate the issue on our end
    • Do not include Personal Identifying Information in the ticket or in the screenshots or files attached to the ticket
    • When having issues with reports, please include the name of the report and the parameters used to run the report
    • Enter one ticket per issue: if you are having an additional issue not related to an existing open ticket, please enter a new ticket as opposed to include the new issue in the existing ticket
    • Review the Knowledge Base articles before submitting a ticket

 

• Privacy and Confidentiality: the AA is in charge of the privacy and confidentiality of their agency data. This includes:
  • Monitor the compliance with standards of confidentiality and collection, entry, and retrieval of data described in the HMIS Policies and Procedures
  • Ensure there is no security violations of the system at their agency
  • Report any security violation reported by other users or noticed by the AA to the HMIS Help Desk

 

• Agency Audits: the AA is in charge of the HMIS annual process of their agency. This includes: 
  • Complete and submit the HMIS Agency Audit Form
  • Conduct Technical Checkings to all the computers and devices used in the agency to access HMIS to verify compliance with technical standards
  • Attend the scheduled HMIS Anual Audit

 

Data Quality

Concept of Data Quality

Data Quality is a term that refers to the reliability and comprehensiveness of the data in HMIS

 

Data Quality Components

We have good data quality in our system when the data we collect is complete, is up-to-date in HMIS, and reflects the real information of the clients in a consistent manner. There are four main components under the concept of Data Quality: Timeliness, Completeness, Accuracy, and Consistency. 

Data Timeliness

• What is it?
  • It refers to the degree to which the data is collected in HMIS and available when it is needed. Data Timeliness looks at how much time passes from the moment data is collected from the client until the moment that data is entered in HMIS.
• Why it is important? 
  • It is important to enter data in a timely manner because it ensures data recorded in HMIS reflects the most current information on the clients served. 
• How it is achieved? 
  • Data Timeliness can only be achieved when the clients' data is entered in HMIS as soon as it is collected.
• Recommendations for reviewing it in HMIS:
  • Data Timeliness thresholds in the OC HMIS is 3 days
  • Review the Data Timeliness Reports once a month. Based on this review:
    • Identify where are the delays in data entry
    • Determine strategies to improve in areas needed
    • Measure and monitor progress over time

Data Completeness

• What is it?
  • It refers to the degree to which all required data is known and documented in HMIS. Data Completeness looks if there is a valid response to all the data elements required for a client. A valid response excludes: Client doesn't know, Client refused and Data not Collected.
• Why it is important? 
  • It ensures data recorded in HMIS represents the population served comprehensively.
• How it is achieved? 
  • Data Completeness is achieved when all required data elements are answered for all the clients in the system. 

• Recommendations for reviewing it in HMIS:
  • Review the Data Completeness Reports at least once a quarter. Based on this review:
  • Identify missing information in data (Blanks and/nor Data not Collected)
  • Identify responses with Client doesn't know and Client Refused
  • Complete data based on this review when possible 

Data Accuracy

• What is it?
  • It refers to the degree to which data entered in HMIS refects the real information of the clients served and the services provided
• Why it is important? 
  • It ensures the reliability and validity of the information recorded in HMIS
• How it is achieved? 
  • Data accuracy is achieved when clients provide real information, and intake staff document and enter information correctly by ensuring congruency in the data. 
• Recommendations for reviewing it in HMIS:
  • Verify that clients' responses are in accordance with the HUD Data Standards. Examples of accuracy errors include:
    • HoH Issues
    • SSN that violates the Social Security Administration rules of a valid SSN (e.g. Individual Taxpayer Identification Number)
    • Incoherence in the Disabling Condition data element
    • Issues with dates (E.g. DoB after Project Start Date, Project Exit Date Prior to Project Start Date)
  • Verify that information provided by clients reflect their real situation: 
    • It is recommended to advise case managers to compare name and DoB information provided by clients against an identification document. This can ensure client names are spelled correctly and DoB is accurate.
    • Emphasize to clients and intake staff that is preferable to enter "Client doesn't know" and "Data not collected" than to enter inaccurate information.
    • Advice intake staff and case managers to guide clients through the entire data collection process so it is ensured that clients understand the questions and their response options.
  • Verify that information recorded in HMIS match what clients reported: 
    • For agencies that use paper forms, it is recommended to have another staff member reviewing the data entered in HMIS against the paper forms.
    • It is recommended to run the Monthly Dashboard, available in HMIS,  to verify the data entered by our agency in the last month represent your clients' information correctly.

Data Consistency

• What is it?
  • It refers to the degree to which data is equivalent in the way it is collected and stored among all agencies that participate in the HMIS
• Why it is important?
  •  It is important to have the same understanding of the clients' information and the way it is collected in order to have accuracy. In other words, data consistency is a prerequisite in order to achieve data accuracy. 
• How it is achieved? 
  • Data ​​​​​​​consistency is achieved when all HMIS users across all agencies have the same understanding of the clients information that is collected and entered in HMIS
• Recommendations for reviewing it in HMIS:
  • ​​​​​​​All HMIS users, especially intake staff and case managers, should review the HMIS Cheat Sheets:
    • ​​​​​​​These sheets provide a detailed explanation of the data elements we collect and provide specific recommendations on how to collect this data.

Common Data Quality Errors

Relationship to head of household (HoH) Error

A household should have one and only one head of household at any given time.

When does this error occur?

The relationship to HoH error commonly occurs when the current head of the household leaves the project while the rest of the members remained enrolled in the project and there is no re-assignation of the HoH.

How to fix or avoid this error? 

When the HoH leaves the projects and the rest of the members remained enrolled: 

1. Designate another member of the household as the new HoH

2. Correct the other members' relationship to HoH to reflect each individuals' relationship to the newly designated HoH

 

Disabling Condition Error

A client's Disabling Condition should be YES if:
a) the client has a physical disability, chronic health condition, mental health problem, or substance abuse problem that is expected to be long term and impair the ability to live independently; or 

b) the client has a developmental disability or has HIV/AIDS

 

When does this error occur?

The disabling condition error occurs when there is an incoherence between the disabling condition selected and the response of the specific disabilities.

How to fix or avoid this error? 

After collecting and/or entering the specific disabilities responses, make sure they are coherent with the Disabling Condition. In case there is any inconsistency, correct the disabling condition. Example: If a client reported they do not have a disabling condition, but then they reported they suffer from a developmental disability, you should change the Disabling Condition to YES.

Date of Birth Error

Recording the clients' date of birth allows us to calculate the clients' age at project start and at any time during the enrollment. Having the clients' age is really important for reporting purposes as it is necessary in order to determine the Household Type. 

When does this error occur?

When Data not Collected is selected for Quality of Date of Birth we are unable to determine if a client is an adult or a child, thus we would not know the Household Type of their enrollment. 

How to fix or avoid this error?

If a client doesn't know or refuses to provide their date of birth, ask them to estimate their age. Enter the year of birth according to their age estimation, and enter January 1st as the estimated month and date, respectively. For Quality of DOB enter "Approximate or partial DOB reported".

 

Data Quality Report Cards

For the publication of the Report Cards, we have three main steps:

Step 1: 211OC sends DQ Corrections to Agency Administrators

• 211OC sends the Data Correction Spreadsheets which highlights data completeness and accuracy errors, as described below:
  • Responses highlighted in yellow: Client doesn't know and Client Refused
  • Responses highlighted in green: Blanks and Data not Collected
  • Responses highlighted in purple: Data accuracy issues

Step 2: Agencies complete DQ Corrections in HMIS

• Action Required from agencies: 
  • Review the Data Corrections Spreadsheet
  • Identify errors that can be corrected
  • Make appropriate corrections to the clients' enrollment in HMIS (See Data Quality Corrections Reference Guide)

Step 3: 211OC publishes Data Quality Report Cards

• The Quarterly Data Quality Report Cards compiled by 211OC assess the level of completeness and accuracy of the Universal Data Elements (UDE) for the system as a whole, as well as each project that contributes data in the HMIS
  • For each UDE it is calculated the percentage of:
    • Missing Responses
    • Responses with Client doesn't Know and Data not Collected
    • Responses that constitute Data Accuracy Errors
    • Valid Responses
  • For each project, the Average Data Completeness Score is calculated by dividing the total valid responses that the project has across all the UDE by the total number of clients that need a response for each UDE. The Average Data Completeness Score is a global indicator of valid responses for each project. 

Data Quality Report Cards Due Dates

The table below specifies the timeframe for each step in the Data Quality Report Cards Process 

 

Takeaways

• The Agency Administrator (AA) is the manager and the steward of Data Quality for their agency.
• The AA should follow the recommendation provided by 211OC in order to monitor their agency's Data Quality.
• The AA should determine additional procedures and processes for reviewing Data Quality that addresses all the data quality components.
• The AA should advocate for the importance of Data Quality with their agency, especially when onboarding new staff.
• The AA should establish data quality as a best practice for the complete cycle of their agency's data entry.