Etaregular Basis.Nder Toor How Ion?
Sub-heads / Indicators / What is the measure?
Binary (Yes/No)
A number
A %age
A grade
Qualitative / In whose domain: State Health Department or Centrally Sponsored Scheme (CSS)? / Is there any standard, rule or a benchmark? / Basis of triangulation/
Verification
HMIS: Capacity
Rank of top officer who is full time/exclusively handling M&E unit/cell at state level.
If there is no dedicated officer, who is handling HMIS?
How much time he/she is devoting? / Grade
Position
Days/week / S / No / Establishment Division of DOHFW
% districts in the state having:
(i) dedicated and trained HMIS personnel;
(ii) clear TORS
(iii) equipped with functioning computer and software / %age / S / Yes
(NRHM guidelines) / HMIS sample
Districts in the state sending monthly and quarterly reports within due date. / %age / S / Yes / HMIS sample
Time lag (in days) in obtaining reports from the district reporting last. / A Number / S / Yes / HMIS sample
HMIS: Review
If the state annual plan is based on HMIS data.If the objectives/targets in stateown plan realistic vis-à-vis current levels reported by HMIS. / Binary (Y/N) / S / No / Perspective Planning Division
Frequency of review of HMIS data for decision making purposes at state level. / A number / S / Yes / Minutes of the meetings
% districts that regularly (at least monthly)
(i) collect, compile and analyze data at district level and,
(ii) reviewdata and take follow up actions / %age / S / Yes / Check in sample districts
What indicators, aside from the national programs, are tracked better in HMIS? (This gives idea of the importance state gives to certain services). / Qualitative / S / No / State annual health reports
Districts that report CHC/FRU performance indicators in their HMIS in last X months / %age / S / Yes
Is there a system for data validation?
If so, what is it:
Do rules exist for data validation before final data approval;
If yes, are these rules systematic? / S / Yes / HMIS sample
Extent of variation between twokey HMIS indicators and independent surveys for year X
(i) institutional deliveries,
(ii) immunization coverage / A number
And
%age / Center and State / No / HMIS, SRS, NFHS
No. of Private clinics/nursing homes/hospitals reporting disease load under IDSPin the state / A number / State / No / IDSP report
Application of Horizontal (within state) mechanisms for sharing of HMIS data across districts/blocks
Application of vertical (across state) ) mechanisms for sharing of HMIS data across districts/blocks / Number of times in a year / State / No / Report of the relevant division.
If there is any formal and regularmechanism for feedback from states to districts based on HMIS data?
If yes, how often does feedback occur? / y/n / State
Broad
Sub-heads / Indicators / What is the measure?
Binary (Yes/No)
A number
A %age
A grade
Qualitative / In whose domain: State Health Department or Centrally Sponsored Scheme (CSS)? / Is there any standard, rule or a benchmark? / Basis of triangulation/
Verification
Procurement Process
% contracts for drugs procurement (above minimal threshold) which were open tender (for one year)
Value of contracts for drugs with open tender as a % of total drug contracts (for one year).
% of open bids in which notification of awards were issued during validity period. / %age / State / Yes / On sample basis
Does a complaint/grievance system exist?
Are details of complaints publicly disclosed?
If so, how?
Contracts with complaints as % of total contracts for procurement of
(ii) drugs, and
(ii) civil works received
% of contracts with complaints for which actions were taken:
ii) drugs, and
(ii) civil works received / y/n
y/n
%age / State / Yes / On sample basis
Average procurement cycle (in months) for
(i) drugs & consumables, and
(ii) medical equipment / Numbers / State / Yes / On sample basis
Procurement Capacity
A separate procurement cell/committee at state/district level / Yes/No / State / No / Establishment division
Rank (hierarchy) of the official who is full time/exclusively for the procurement cell/committee. / Qualitative / State / No / Establishment division
Number of full-time staff members with formal degrees to handle procurement in the state health department.
Number of full-time staff members with >2 years experience in handling procurement
Value of procurement conduction by the cell as % of total value of procurement done by state health department. / Number / State / No / Establishment division
Existence of at least one biomedical engineer in the state health department exists (this is higher end procurement capacity indicator). / Binary / State / No / Establishment division
Contracts in which external experts took part in the bid evaluation process in case of sophisticated medical equipment (> 50 lacs).
External to health department?
External to government? / %age / State / No / Minutes of technical bid evaluation committee
Procurement: Quality Assurance
Does a pre-dispatch inspection system exist for pharmaceuticals?
pre-dispatch inspections as % of total dispatches for pharmaceuticals. / %age / State / Yes / Inspection Reports
Extent to which technical specifications for medical equipment included
(a) compliance certificate from laboratories (WHO accredited or NABL) or
(b) performance standards as requirement / %age / State / Yes / On sample basis
An indicator on (number of instances of?) counterfeit drugs during last fiscal year.
Broad
Sub-heads / Indicators / What is the measure?
Binary (Yes/No)
A number
A %age
A grade
Qualitative / In whose domain: State Health Department or Centrally Sponsored Scheme (CSS)? / Is there any standard, rule or a benchmark? / Basis of triangulation/
Verification
Contract Management
% contracts for civil works above Rs. 50 lacs, in which more than 15 variation/ change orders were issued during year X. / %age / State / No / Sample of contracts
Are medical equipments issued valid AMC?
Medical equipment costing Rs. 30 lacs or more, having any valid AMC during year X. / y/n
%age / State / Yes / Sample purchase orders
Installation of medical equipment valued 10 lac and above delayed by over 3 months with reference to the agreed contract.
Percentage of contracts above certain value. / %age / State / No / Sample of certificates of receipt and installation
Is a procurement audit of process compliance conducted on a regular basis?
Is there evidence of timely action taken to address any shortcomings highlighted in the audit?
Is a procurement regulation audit conducted on regular basis?
Is there evidence of timely action taken to address any shortcomings highlighted in the audit?
% of maintenance contracts in which at least a portion of payments are linked to quantifiable performance indicators.
% of non-clinical service contracts (cleaning, food, laundry, etc.) in which at least a portion of payments are linked to quantifiable performance indicators. / Binary / State / No / Sample of procurement audit reports.
For action taken, check the effect of actions on a sample basis.
Broad
Sub-heads / Indicators / What is the measure?
Binary (Yes/No)
A number
A %age
A grade
Qualitative / In whose domain: State Health Department or Centrally Sponsored Scheme (CSS)? / Is there any standard, rule or a benchmark? / Basis of triangulation/
Verification
Supply Chain Management
% districts reporting on stock-position of drugs to state health department on at least a bi-yearly basis. / %age / State / Yes / Sample districts
Stock-outs for vaccines and essential drugs at PHCs/FRU level reported to state on a regular basis. / %age / State / No / Stock registers on sample basis
Average cycle time for stock replenishment in district store / A number / State / No / Indent registers at district stores
Average monthly consumption of essential drugs and vaccines vis-à-vis average volume per replenishment/refill.
Inventory management indicator. Is there a system to monitor monthly consumption vis-à-vis replenishments of
Essential drug and Vaccines? / A number / State / No / Stock registers (for consumption) and indent register (for volume)
System exists to record to drugs lost/destroyed/expired due to any reason at district stores.
what it the incidence of such loss? / y/n
A number / State / Yes / Sample districts
etaregular basis.nder toor how ion?
Sub-heads / Indicators / What is the measure?
Binary (Yes/No)
A number
A %age
A grade
Qualitative / In whose domain: State Health Department or Centrally Sponsored Scheme (CSS)? / Is there any standard, rule or a benchmark? / Basis of triangulation/
Verification
Financial Management: Audit / Districts submitted SOEs/UCs in timely matter according to preset deadlines. / %age / State / Yes / Financial Monitoring Reports (FMRs)
Extent of variation between audited and reported expenditure at state level during the last available year.
Salary
Non-salary / %age / State / No / FMR and audit reports
Districts have full-time accounting/ finance officer at district level. / %age / State / Yes / State NRHM Report
Extent of delay (in months) in the last internal audit report and external audit report at state level / A number / State / Yes / External and Internal Audit Report
% of contracts for medical purchases valued more than Rs. 10 lacs, if the suppliers/ contractors receive payments on time.
(i) drugs
(ii) equipment / %age / State / Yes / A sample of contracts
% contractors receive payments on time in most recent year for:
(i) Annual Maintenance Contracts (AMCs)
(ii) Comprehensive Maintenance Contracts (CMCs) / %age / State / Yes / A sample of contracts
Broad
Sub-heads / Indicators / What is the measure?
Binary (Yes/No)
A number
A %age
A grade
Qualitative / In whose domain: State Health Department or Centrally Sponsored Scheme (CSS)? / Is there any standard, rule or a benchmark? / Basis of triangulation/
Verification
Service Delivery PPPs
If service contracts (e.g. mobile clinics, contracting out of facilities, contracting in private providers etc.) initiated on PPP basis.
The total value of those contracts during particular period. / A number (of contracts and their total value) / State / No / FMR and Department’s Annual report
In such service contracts, are there quantifiable deliverables or outputs in the TORs. / Binary / State / No / Contracts on sample basis
In service contracts with private agencies to whom payments were made, % in which inception and progress reports were received. / %age / State / Yes / Such reports on sample basis
% of service contracts in which at least a portion of payments were linked to quantifiable performance indicators. / %age / State / No / Contracts on sample basis
Percentage of service contracts that are monitored on regular basis.
How often, is a report issued?
Percentage of service contracts that have been cancelled last year.
% service contractors receive payments on time in most recent year. / state / No
Yes
Is there any grievance redressal mechanism in place for service contracts? / Yes
Broad
Sub-heads / Indicators / What is the measure?
Binary (Yes/No)
A number
A %age
A grade
Qualitative / In whose domain: State Health Department or Centrally Sponsored Scheme (CSS)? / Is there any standard, rule or a benchmark? / Basis of triangulation/
Verification
Human Resource
For doctors and nurses, are there a clear set of uniform rules/criteria that define:
(i) recruiting
(ii) hiring
(iii) retention/tenure
(iii) postings
(iv) transfers
(v) sanctions/suspension/dismissal
(vi) redress
(vii) performance assessment
(viii) promotion
If yes, the extent to which these rules are followed in each of the category for which there exist rules/criteria / y/n in each case / State
HR Department Capacity
If there is dedicated HR management Unit?
Position/level of top officer who is full time handling health HR functions at state level.
If no dedicated officer, who is handling HRM and how much time he/she is devoting.
Number of full time staff working in HR department / y/n
Qualitative
Qualitative
A Number / State / No / Establishment division
ACRs are prepared.
Number of years of computerized Annual Confidential Reports (ACRs). / y/n
A number / State / No / Establishment division
Length of the last recruitment cycle (in months)
(i) physicians
(ii) nurses
From job advertisement to the joining of selected candidates / A number / State / Yes / Establishment division
An indicator on mainstreaming of state consultants into regular cadre (rules on this and number of such cases).
Personnel Management Information System (PMIS) exist. / y/n / State / No / Observation/
Inspection
If a system exists to cross-check and update personnel record and payroll commitments
How frequently it is done? / A number / State / No / Records in the establishment division
For what levels of staff are there no written job description or the job description not revised in the last 5 years
Job descriptions are clear for senior staff (MO and above) / Qualitative / State / Yes / Establishment division
Of doctors ‘appointed’ (issued offer letters), % who ‘joined’ the service. / %age / State / No / Establishment division
% of PHCs without physicians.
Average turnover rate of MOs in PHCs
-Urban
-Rural
% Specialist vacancies at District Hospitals / State / Yes / NRHM report
If incumbent paramedical staff went on at least one training in the last two years.
(i) nurses
(ii) AMNs
(iii) ASHAs
(iv) MOs / %age / State / No (based on program plans) / Program division on sample basis
For staff at all levels, how many poor performers were identified, and against how many were any disciplinarian action taken. / A number / State / No / Records of CMO office in sample districts
What are the major criteria for determining promotions?
-years of service
- tests
- assessment by superior
- other / Qualitative
Practices to attract and retain physicians in undesirable rural locations:
-Bonuses
-Special benefits
-Post graduate seats
-Other
Any other performance-related-pay practices?
-Physicians
-Nurses / Qualitative
An indicator on staff union, who are its members and what’s its mandate?
An indicator on wage arrears (in weeks) relating to state consultants (who are different from NRHM consultants)
Is health staff subject to at least an annual formal performance appraisal?
What is the source of information in the performance appraisal?
Is there a written record of each civil servant performance appraisal?
Existence of independent redress mechanism.
If so, what is it?
How many cases started in last 3 years?
How many cases decided in last 3 years? / y/n
Most typical means for promotion in the agency?
Is there an adequate code of conduct?
Is it enforced?
If so, how? / y/n
y/n
Qualitative / state
Is there a body with the mandate to undertake independent decisions on the fairness of HR policies?
Does this oversight body have sufficient authority to enforce decisions?
Does the oversight body have M&E capacities? / y/n in each case
Staff resigning as % of total in most recent 3 years
(i) MBBS doctors
(ii) specialists
(iii) nurses / %age
%age
%age / state
Is there a formal unified performance appraisal system?
How often is it applied (quarterly, annually, biannually)?
Is there recourse for appeal for promotion grievances?
Is there a written process for conducting disciplinary hearings? / y/n
Qualitative
y/n
y/n / State
Are there specific training budgets identifiable in the state budget? / y/n / State
Annual turnover rates of senior staff for last 3 years / A number / state
Data collected on physician absentee rates / %age / State
Are all major organizational units (department, hospital, etc.) required to specify objectives in writing?
Are all major organizational units required to track and report systematic evidence on their progress toward achieving those objectives? / y/n
y/n / State
% of staff in last 3 years
-disciplined
-suspended
-dismissed / %age in each case / state
Regular audits performed on:
-recruitments
-promotions
-postings
-transfers
-suspensions / y/n in each case
Broad
Sub-heads / Indicators / What is the measure?
Binary (Yes/No)
A number
A %age
A grade
Qualitative / In whose domain: State Health Department or Centrally Sponsored Scheme (CSS)? / Is there any standard, rule or a benchmark? / Basis of triangulation/
Verification
Planning
If the state health goals to be achieved over next 5 years clearly specified. / Binary / State / Yes / Current Annual report
Health goals that the state has achieved in the last 5 years. / Qualitative / State / Yes / Annual reports
How long (in time period) is the policy planning cycle.
The difference in state budget of the proposed plan and the executed plan. / %age / Pertains to CSS / No / NRHM Record of proceedings (ROP)
Districts that revised their proposed plan based on the budget that actually got approved. / %age / Pertains to CSS / No / ROPs of sample districts
If state own health plans linked to NRHM program implementation plans. / Binary / State / No / Plan document
Districts that submitted district action plans to the state on time. / %age / Pertains to CSS / Yes / NRHM Report
Districts whose health plans reflected convergence with wider determinants of health like drinking water and sanitation. / %age / Pertains to CSS / Yes / District PIPs of sample districts
Districts that have fully functional district health societies with full time management officer in place / %age / State / Yes / NRHM Report
Results indicators involving outputs or outcomes included and clearly defined in plan / y/n / State
The extent to which planning exercise is based on systematic situational and information analysis for programs, services and activities. / y/n / State
Existence of formal planning unit / y/n / State
Some notion of financial envelope provided prior to formulating plan / y/n / State
Decentralized planning
District Planning Committees (DPC) under the PRI involved in finalization of the district health plans. / %age / State / Yes / Plans in sample districts
Blocks that conducted block level public hearings events i.e., ‘Jan Sunwai’ or ‘Jan Samvad.’ / %age / State / Yes / Relevant department
Blocks that prepared health plans by themselves prior to the preparation of district health plan. / %age / State / Yes / NRHM report
PHCs that have Rogi Kalyan Samities [RKS] constituted within the overall Panchayati Raj framework. / %age / State / Yes / NRHM report
How often are meetings held at the District level with PRI members to understand their perception of health needs and role of PRI in addressing these / A number / State / Yes / NRHM report/minutes of District health society
Broad
Sub-heads / Indicators / What is the measure?
Binary (Yes/No)
A number
A %age
A grade
Qualitative / In whose domain: State Health Department or Centrally Sponsored Scheme (CSS)? / Is there any standard, rule or a benchmark? / Basis of triangulation/
Verification
Budgeting
Extent to which the budgetary exercise for the annual plans linked to state 5-year plan. / Qualitative / State / Yes / Relevant documents
If the state plan allocation to the health sector linked to sector strategies. / Qualitative / State / No / Plan documents and sector strategies
Share of health allocations not governed by fixed allocation rules (discretionary budget) / Percentage / Pertains to CSS / Yes / FMRs
Implications of state plan allocations on the recurrent expenditure in the subsequent plans fully taken into account while making those allocations. / Qualitative / State / No / Key informants in Finance department
Extent of deviation between state budget allocations and actual expenditure during the last fiscal year. / Percentage / State / No / Budget documents
Extent of deviation between the central health allocations and the actual expenditure during the last fiscal year. / Percentage / Pertains to CSS / No / Budget documents
Share of state capital budget for health that was actually spent. / Percentage / State / No / Budget documents
Which of the following types of budgetary and financial information made available to the public:
(i) Annual budget information;
(ii) year-end financial statements;
(iii) external audit reports; and
(iv) contract awards in excess of approximately Rs. 50 lacs. / Qualitative / State / Yes / Physical verification
If quarterly reports on budget execution prepared at state level. / Binary / State / Yes / State treasury office
Ratio of Plan/Non-Plan and Capital/Revenue Budget / %age / State / No / Budget documents
Proportion of intended funds to districts actually reach districts.
Broad
Sub-heads / Indicators / What is the measure?
Binary (Yes/No)
A number
A %age
A grade
Qualitative / In whose domain: State Health Department or Centrally Sponsored Scheme (CSS)? / Is there any standard, rule or a benchmark? / Basis of triangulation/
Verification
Regulation
Health institutions (public and private) registered under Medical Termination of Pregnancy Act/Pre-natal diagnostic tests (Prevention) Act/Clinical Establishment Act / A number
Percentage / State / Yes / CMO office in the sample districts
Public and private health institutions in the state certified by PCB (Pollution Control Board) for adhering to Hospital Waste and Environment Protection Act / A number
Percentage / State / Yes / State Pollution Control Board
If the regulatory rules for private health care providers implemented differentially in comparison to the public health care providers.
Formal mechanism for inter-departmental coordination. / Binary / State / Yes / Coordination mechanism
Broad
Sub-heads / Indicators / What is the measure?
Binary (Yes/No)
A number
A %age
A grade
Qualitative / In whose domain: State Health Department or Centrally Sponsored Scheme (CSS)? / Is there any standard, rule or a benchmark? / Basis of triangulation/
Verification
Feedback Mechanism
Do mechanisms exist to systematically obtain client input on satisfaction with health services?
If so, what is the mechanism
How frequently is it applied?
Do mechanisms exist of public disclosure of
(i) financial data and
(ii) system performance exist
Is so, what is the mechanism? / Binary (Y/N) / CSS and State / Yes / --Policy/proposed bill put up in public domain
--community monitoring reports
Right to information (RTI) cell in place, disclosed to the public, request for information furnished within stipulated time / Binary (Y/N),
Qualitative / State / Yes / Observation/physical verification
Each of the indicators above will lead to a set of logical questions that will provide basis of collecting information on the desired indicator which is only indicative at this moment but can be refined when the questionnaire is administered.