Two randomized controlled trials of the SEEK approach yielded promising findings with lower rates of child abuse and neglect and harsh parenting. There remains an ongoing need, however, to further evaluate SEEK in different settings. And, the MOC 4 or Performance Improvement (PI) Activities require demonstrating how SEEK has improved the care you provide in your practice.

It is necessary to carefully choose a specific goal(s) and how to measure whether these have been met. There are several practical considerations tied to what resources are available. For example, auditing a sample of charts to assess the rates of screening for targeted problems is straightforward. Assessing whether parents engaged in recommended services requires more work, and assessing whether they improved, for example, with lower depression scores involves still greater effort. The SEEK team will help you set a goal(s) and a method for measuring outcomes of interest. We have developed several brief SEEK tools with guidelines for the evaluation. There are also good tools others have developed, such as for depressive symptoms.

Some goals pertain to health professionals; their attitudes, awareness, level of comfort, perceived competence and practice behavior regarding the targeted problems. The assessment can be done by: 1) the self-report SEEK Primary Care Professional Questionnaire (PCPQ), 2) review of Medical Records to assess the rate of screening, the rate of positive screens, and the types of interventions, and 3) direct observation during a few well child visits. In addition, the PCP Experience with SEEK survey assesses PCPs’ experience with the approach.

Some goals pertain to parents. Their receipt of and satisfaction with services, such as for substance abuse, can be assessed, as well as their satisfaction with their child’s health care. There are also measures to assess possible improvement in the targeted areas – depression, domestic violence, food insecurity, substance abuse, harsh discipline and major stress.

Some goals pertain to children. Given their young age and the limited duration of most projects, improvements in their health, wellbeing, development and safety are difficult to demonstrate. There are, however, several tools we and others developed to assess these areas, and there are related ICD 10 codes that can be readily abstracted from EHR data.

Another goal pertains to the experience of office staff in implementing SEEK. The SEEK Office Staff survey measures staff perceptions of implementing the approach.

Finally, there may be interest in conducting a needs assessment of the community you serve related to the prevalence of certain problems. Accessed via the SEEK Parent Questionnaire-R or Re. Such information can be the basis for improving linkages with community resources and for advocating for new resources if needed.

Sample Goals

These are examples of goals for a SEEK MOC 4 or Performance Improvement project:

• To increase the rate of screening for targeted psychosocial problems during checkups of children 0 - 5, by 20%.
• To ensure that an action is taken in 75% of instances when a problem appears likely
• To improve PCPs' attitudes, awareness, level of comfort, perceived competence and practice behavior regarding the targeted problems, by 20%
• To increase the rate that families needing services receive them, by 20%
• To increase parent satisfaction with their child's health professional, by 20%
• To reduce rates of parental problems, such as depressive symptoms, by 20%
• To improve children's health, wellbeing and safety, by 20%

Measures for Evaluating SEEK

1. The SEEK Medical Records Evaluation Form

This form includes guidance for abstracting information from children’s medical records regarding screening for and addressing the targeted problems, during regular checkups. Documentation can capture screening, the result of screening (i.e., positive or not), whether an action occurred, and the type of action - for each targeted problem, at a visit designated for screening.

If interested, medical records also can be reviewed for specific problems/outcomes of interest, such as ingestions, failure to thrive, injuries, child abuse, child neglect, delayed immunizations, and non-adherence to healthcare recommendations.

2. The SEEK Parent Questionnaire-Re (PQ-Re)

The PQ-R / PQ-Re screens for the targeted psychosocial risk factors: parental depression, substance use, major parental stress, intimate partner violence, food insecurity and harsh punishment. In addition to assessing possible problems facing individual families, data can be aggregated to assess the prevalence of problems in the population served by a practice (i.e. a needs assessment).

One or two questions screen for each risk factor and are answered Yes/No. A "Yes" to either question (when there are two) is considered a positive screen, needing further assessment and possible intervention by a clinician.

The PQ-R / PQ-Re is completed by the parent or primary caregiver in advance of selected regular checkups for children aged 0 - 5 (e.g., 2, 9, 15, 24, 36, 48 and 60 months). It has 25 yes/no questions and takes 2-3 minutes to complete. For more information, check out the PQ-Re information sheet on the SEEK Materials page.

When is the Screen Positive?

• Food Insecurity: A ‘Yes’ to either food related question.
• Harsh Punishment: A ‘Yes’ to the “slap or hit” question or “….. child is difficult …..”
• Major Stress: A ‘Yes’ to “child is difficult”, “more help” or “extreme stress”.
• Depression: A ‘Yes’ to “feeling down” or “little interest”.
• Intimate Partner Violence: A ‘Yes’ to either of the two questions.
• Substance Abuse: A ‘Yes’ to either question.

Possible Concerns

Regarding a false positive screen:

• A screen is just a screen. A brief assessment should quickly help clarify whether this was falsely positive.
• Be careful not to convey a diagnosis, for example, of depression. Further evaluation should clarify that.

Regarding a false negative screen:

• It’s inevitable that some problems will be missed. There are many reasons why someone may choose not to disclose a problem.
• It’s possible that by asking the question(s), you've shown your interest, and sown a seed. They may disclose in the future.
• Parents who choose not to disclose are probably not amenable to intervention at this time.

3. The SEEK Primary Care Professional Questionnaire (PCPQ)

The PCPQ assesses a primary care professional's (PCP’s) awareness, attitudes, level of comfort, perceived competence and self-reported practice behavior regarding addressing targeted psychosocial problems - parental depression, substance use, major parental stress, intimate partner violence, food insecurity and harsh punishment.

There are 5 brief case vignettes, each followed by 8-12 statements that are rated on a scale. Scores can be derived for the thematic scales (e.g., level of comfort) that cut across subject areas as well as for each subject area. The latter cover aspects related to the subject (e.g., parental depression), such as the PCP's attitudes toward and perceived competence in addressing the problem. We plan to develop scales based on the latest version of the PCPQ as part of the current NIH-funded study.

The PCPQ is completed by the PCP, in 10-15 minutes. It can be administered prior to and after training and at later times to track changes.

4. SEEK Primary Care Professional Experience with SEEK

This questionnaire assesses the PCPs’ perceptions about:

• The SEEK approach
• The SEEK PQ-R
• The SEEK Guidelines
• The SEEK Responses to Barriers
• The SEEK Parent Handouts
• The SEEK Website
• The SEEKonline software
• Behavioral Health Professionals (including Social Workers)

5. SEEK Office Staff Experience with SEEK

This questionnaire for primary care office staff evaluates perceptions about their role in implementing SEEK and the processes of implementing the approach in the primary care practice.

6. Parent Experience with Child's Healthcare

This questionnaire was adapted to assess parent satisfaction with their child's PCP. The questionnaire is completed by the parent in approximately 3 minutes.

Falvo, D. R. & Smith, J. K. (1983). Assessing residents' behavioral science skills: Patients' views of physician-patient interaction. Journal of Family Practice, 17, 479-483.

7. The SEEK PCP Observation Checklist

Ideally, self-report questionnaires are complemented by direct observation to assess PCPs’ practice behavior. This checklist helps assess PCPs’ efforts regarding the targeted problems, during regular checkups for children 0 - 5. Efforts are identified (Yes/No) regarding screening, assessing, and intervening, and the nature of the intervention - for each problem, at each visit being observed, and designated for screening. One component is to time the entire visit, and, the time spent on social determinants of health.

Guidelines are available for assessing the observations. This should be done by someone familiar with pediatric practice and can be an interesting medical student project. Generally, we recommend observing three checkups per PCP for an adequate sampling.

The time required is the duration of the visit, as well as time between eligible visits.

8. The SEEK Family Services Questionnaire

This questionnaire elicits input from parents as to whether they received certain commonly needed services in the past month, their rating of services received, and whether they are currently interested in getting the service. In addition to guiding interventions for individual families, this questionnaire can serve as a needs assessment regarding services commonly desired.

It consists of 15 items and takes 2-3 minutes to complete.

9. The SEEK Child Protective Services (CPS) Record Review

CPS data can be used to assess possible child abuse and neglect. This could be done at the individual, family or county level. The SEEK Project team is available for guidance on this process.

Other Measures

SEEK has utilized other standardized measures in several areas:

• Child: behavior (Child Behavior Checklist), development (SEEK Child Development Screen), health problems (review of medical record)
• Parent: discipline (Parent Child Conflict Tactics Scale), parenting (Parent Sense of Competence Scale), depression (Beck Depression Inventory II), substance use (CIDI)
• Family: domestic violence (Conflict Tactics Scale), family functioning (Family APGAR)

There are many other measures available to assess these and related areas. The SEEK Project team is available for guidance on such measures.