Book Bag Program Evaluation Information
This information was taken from San Mateo County Early Learning Project, Project Summary/Abstract, 2003. Authored by the Center for Venture Philanthropy based on several independent evaluative studies.

The impact of the Raising A Reader® Book Bag Program has been measured in health home visiting settings, randomly selected small family child care homes and child care centers, and in more formal Head Start settings.  The following demonstrates the impact of the Raising A Reader program in San Mateo County.

Parents:

In Spring 2000, a survey of a random-sampling of 718 San Mateo County families in the Raising A Reader program resulted in a 91.2% return rate.  After 3 months in the program, an independent evaluator reported there was an increase of 15 percentage points in the number of families reading frequently (3 times/week or more).  In low-income Spanish-speaking families, the increase was 21 percentage points. 

The San Mateo County Health Department performed an evaluation in 2000 to measure the impact of RAR in health home visiting nurse programs. First time mothers showed a 70% increase in reading daily to their children. Also, an increase in library usage was significant. Families visited the library 41% more than those not using the program.

In August 2001, an evaluation assessed the impact of Raising A Reader with its new “Advanced Read-Aloud Curriculum”, “Read Aloud” Parent Video and strategies for increasing library attendance. An independent evaluator found a 33% growth in the percentage of Head Start parents reading to their children frequently each week (three or more times/week) and a 59% growth in the percentage of mono-lingual Spanish families’ reading behavior.  There was a 337% growth in the percentage of Spanish-speaking parents taking their preschool children to the library at least once a month and an 86% growth across the whole spectrum of children.

Kindergarten Readiness:

Completed in September 2001, an evaluation demonstrated that Raising A Reader low-income children entered kindergarten more prepared to learn than children of a similar socio-economic background.  Pre-literacy skills were assessed in 100 Head Start-Raising A Reader four-year-olds entering kindergarten, using the nationally-normed Head Start FACES Test.  Raising A Reader children (in the program for 8 months or more) scored more than twice the national norm for Head Start students in knowledge of print concepts and story comprehension.

In 2002, kindergarten teachers across San Mateo County, in blind scoring, gave higher kindergarten readiness scores to children who had participated in Raising A Reader, compared to children from similar socio-economic backgrounds. 

Child Care Providers:

An independent evaluation measured satisfaction of child care providers participating in a small 1999-2000 pilot.  The providers confirmed that the Book Bag Program was easy to use and very exciting to providers, children and their parents.  Ideas for improvement were incorporated into a new Teacher’s Guide and Teacher Idea Book. 

Community Reading Van Evaluation Information
(previously called Reading on the Road)
Taken from San Mateo County Library Reading on the Road, First 5 San Mateo County Year End Evaluation Report, authored by Kathleen Fletcher, and independent evaluator, December 30, 2004.

Evaluation Findings

1.  Progress toward desired results/outcomes

Outcome 1:  60% of WIC parents who received a library card through Reading on the Road indicate that they are more aware of the importance of early literacy and the services provided by libraries as a result of the program.

Library staff attempted to reach through a telephone survey the parents who had received a library card through Reading on the Road and who were still assumed to be at the same address because a mailing to them had not been returned to the library by the post office.  Of those, staff was able to complete 58 surveys, 51 in Spanish and 7 in English.  Forty-six of those 58 (79%) indicated that the information they heard at WIC increased their knowledge and helped them change what they did about reading with their children.  Table 1 shows the ways in which they indicated that they changed.

Table 1
Number Indicating Changes in Reading Behavior
After Reading on the Road Visits to WIC

Items under the “other” category included understanding that children can start to learn at an early age, having more books around, trying to help children more with their homework, and talking to children about books.  One parent commented that her son is more interested in books now.

In addition to increased knowledge about reading, 43 parents (74%) indicated that they now know more about what libraries have to offer.  Prompted to name one thing they had learned, respondents said the following:

• Learned that there were computers for public use (8 responses)
• Learned about story times for children (8 responses)
• Learned about all the services libraries provide (6 responses)
• Learned about materials in Spanish (6 responses)
• Learned about checking out movies and videos (4 responses)
• Learned about children’s programs (3 responses)

Other things listed included:

• If you can’t find information, staff will help you.
• I can place holds on books.
• There are Spanish-speaking staff.
• I can check my account on the Internet.
• There are ESL books and tapes.
• I can use libraries other than my local library.

Outcome 2:  75% of WIC site staff report that they have learned new skills to build literacy in young children and are using them at the WIC site.

Twelve WIC staff members, representing five WIC sites (East Palo Alto, Redwood City, Half Moon Bay, San Mateo, Daly City) completed a survey, all of whom had attended meetings where someone from Reading on the Road had talked to clients.  All but one (92%) said they had learned new information on literacy and that they were using that information in their work.  Examples of things they learned and ways they were using them include the following:

• To encourage parents to read to their kids.
• How important it is to read to children.
• More clients are joining the library and reading to their kids at home.
• There are books available to kids of all ages.  It is never too soon to introduce a child to a book.
• Letting clients know how easy and convenient it is to obtain a library card and to get books and other library services.
• Encouraging clients to go to the library and starting to read books to their children.
• Promoting reading.  Keeping books available for children in our offices.
• Promoting early exposure to books for babies and children.

Outcome 3:  50% of parents who received a new library card report that the amount of time they spend reading to their child at home has increased as a result of the program.

As outlined in Outcome 1, 28 of the 58 phone survey respondents (48%) indicated that they read more often to their children.  One additional parent respondent indicated that she talks to her children more about books, and one said she has more books around the house; both of these responses imply increased reading with their children.  If counted, the percentage increases to 30 of 58 (52%).  

Outcome 4:  There is increased use of the library by families served by Reading on the Road as measured by new library cards and books checked out under those cards.

A total of 270 new library cards was issued by Reading on the Road to parents at WIC sites during the second year.  With 243 having been issued during the first year, a total of 513 library cards has been issued since the beginning of the project.  Library records show that of this 513, 151 clients (29%) used their library cards this year.  This is a major increase from the first year, when only 9% had used their card. 

Outcome 5:  The number of books in children’s homes increases.

Records show that 2,289 books were given away to children and parents at WIC sites during visits of the Reading on the Road van.  An additional 14,117 books provided by Reading on the Road were given out by health clinic staff (see Outcome 7).  This equals a total of 16,406 free books distributed to low-income parents.

Outcome 6:  WIC site staff visited by Reading on the Road demonstrate satisfaction with the site visits and services offered.

All of the 12 WIC site staff surveyed (as described in Outcome 2) indicated that they were satisfied with the services provided. Their comments below show their enthusiasm and the reasons for their satisfaction.

• The Reading on the Road staff help us to make parents aware of how important is reading.
• The children at our sites don’t always have the opportunity to attend the library and they seem very interested with the activities at the offices.
• Children really enjoy looking at the books and having stories read to them in the WIC office.
• Sometimes people don’t make an effort to visit the library.  The library coming to WIC is very much appreciated by everyone.
• Library cards are given out at every visit and there is always at least some interest in what the library has to offer.
• Clients enjoyed books that are checked out or the books they receive for free.
• Clients are able to obtain library cards.
• Many of our clients do not know that the library is free, where it is located, or what other services are offered such as story time.
• The lady who comes by and reads to the children, talks to the parents, signs people up for library [cards] does a great job.  Very good enthusiasm.
• It is a pleasant collaboration, it adds to our services, if fits well into our program.

From the responses quoted above, it can be seen that WIC staff welcome the program with enthusiasm and definitely believe it is benefiting their clients.

Outcome 7:  Health clinic staff and pediatricians indicate satisfaction with the services offered.

Intervention at health clinics involves providing children’s books for the waiting rooms, reading to children as they wait for their appointment, and providing books for doctors to give to parents during well-child visits.  Satisfaction surveys were completed by health clinic staff at six health clinics served by the program:  Ravenswood Family Health Center, North County Health Center, Willow Clinic, San Mateo County Health Center, Coastside Family Medical Center and Fair Oaks Children’s Clinic.  A total of 26 written surveys was collected, and an additional 12 in-person surveys were completed through interviews.  Of the 26 who filled out a written survey, all indicated that they thought the program was helpful to patients.  Some of their answers to how it was helpful included:

• It introduces parents to books and stresses the importance of using books with children.
• Lots of my patients have no books.
• Parents love the books.  Many of our moms carry them in their purse to read while they are waiting places.
• Our patients love this program.  We have a patient base [such] that these are the only books they get and they love them.
• Encourages kids to read, positive aspect of office visit (makes up for the shots, etc.!)
• It is both good for the parent and good for the child to be presented with a book by a person they esteem.
• It gives the parent and child an opportunity to bond.
• I believe it is very helpful in that parents are spending time with their child and are promoting a crucial skill (reading) a person needs throughout life.
• Some parents can’t afford to purchase books for their kids.
• Giving books to kids encourages parents to read to them.
• It gives the parents bonding time with their children; it gives the child something to read—and how excited they are to receive the book from the providers!
• Children really enjoy the book, are very interested, and it shows when they receive the book by the way they look at it.
• A great vehicle to discuss the importance of early childhood literacy.
• Stress is put on reading and the importance for future education.
• More children know about books and like books at an early age, which makes them more interested in books when they’re older.
• Giving out books shows that we think reading is important for the well being of the child.  Giving a book during a visit gives providers an opportunity to demonstrate ways parents can interact with their children and books.
• Makes them aware that books and reading to children are important and that they should start early.

In response to a question asking them whether they were satisfied with how the health clinic intervention program is running, all but one of the 26 respondents indicated they were satisfied.  That one person checked “not sure” and wrote that he or she didn’t know how the program is supposed to be running.  In answer to why they were satisfied, respondents said the following:

• The children love the books, and the parents notice how their baby’s eyes “light up” when I hold the book in front of the baby.
• Many of our parents have limited resources and little exposure to books.  It really shows the parents that books are an important part of childhood learning and development.
• We can give books to kids who have none, teach parents about reading/talking about books.
• It seems to flow very nicely and the selection is age-appropriate and fun for the kids.
• At each well-child visit, parents receive a book for their child, which is great for those parents that cannot afford to buy books.
• I’ve noticed that when the children are coming out of the exam rooms, they are enthused with their books, and parents are looking forward to reading to them.
• The program is great from what I see.  Our clinic just needs additional space to store our books.
• The children get age-specific books.

A number of respondents indicated that they appreciated the work of the Reading on the Road coordinator.  Such comments included:

• Books are always available for office visits, coordinator at our site is responsible, enthusiastic, great to work with.
• [The coordinator] is very helpful and knowledgeable about the program.  She keeps our supply stocked.
• Great support from [the coordinator], she is always looking for ways to improve.
• The book lady always comes in and checks our stock and keeps us stocked with a good supply of books.
• [The coordinator] visits our facility 1-2 times a month and checks with staff for any questions or problems and consistently brings us new books.
• [The coordinator] is really friendly and responsive to our needs.

There were several comments on how Reading on the Road could improve their services.  One respondent asked for more books dealing with puberty in Spanish, four asked for books for older children, four indicated they would like books restocked more often, and three indicated they would like more volunteers to read to children in the waiting room.

The last question on the written survey asked for any other comments, and most were reiterations of praise and appreciation for the program.  The following three comments, however, added a new twist.

• With the serious warning that TV is damaging to children’s brains, especially for those under 2 years old, families will need more books at home.
• Kids really enjoy seeing and reading the books while waiting.  Before this, all there was [in the waiting room] were magazines and health education pamphlets for adults.
• I’ve noticed that the program works both ways.  Some of the parents are even practicing their English skills as they read and enrich their child with a book.

2.  Additional findings

a.   The phone survey of parents who had received a library card during a Reading on the Road visit to their WIC site asked whether respondents had ever visited their local library and, if so, when the last time that they visited was.  Thirty-five of the 58 respondents (60%) indicated that they had visited the library:  16 within the last month, 15 within the last six months, and 4 within the last year.  Respondents were also asked to indicate what they did at the library during their last visit.  Table 2 shows those answers.

Table 2
Parent Activity on Last Visit to Local Library

One additional parent indicated that her library activity was to pick up a gift T-shirt and free book.

The 21 people who indicated they had not visited the library were asked what the hardest thing was about getting to the library.  Seven individuals indicated they did not have time, four indicated they did not have transportation, and three indicated that they did not know the location of their local library.  Others mentioned work, living in a remote area, not reading very well, pregnancy, and children’s health problems as barriers to their visiting the library.

b.   The survey of WIC staff described in Outcomes 2 and 6 above also asked respondents for “Other comments.”  While somewhat repetitive of comments in response to the question of satisfaction with the services, the comments below illuminate additional benefits of the program for clients as seen by the WIC staff. 

• Now at WIC because of your program, our participants will have a chance to know how important reading is for children.  Also parents will know of services related to reading like reading time and/or story times.
• Have a library mascot such as an animal or puppets that children can associate the library to.  Keep coming to WIC sites, our clients love [the coordinator] and her books.
• We really appreciate the services.
• It’s wonderful!  I love the free books and the story telling visits.

Several of the comments praise the work of the Reading on the Road coordinator, who makes the visits to the WIC sites.

• Thank you to [the Reading on the Road coordinator] for her kindness.  Her eagerness to learn Spanish and her ability to communicate with the Latino community is appreciated by my staff at WIC.  Her warm and friendly attitude makes and encourages the population to go and visit their library and utilize the resources available to them.
• [The Reading on the Road coordinator] is very passionate about the importance of literacy and the benefits of reading.  She communicates effectively with Hispanic families despite still learning Spanish.  She always helps and informs as many people as possible.
• We truly enjoy and welcome [Reading on the Road coordinator]’s charisma.

c.    A finding from the phone survey of WIC clients who received a library card during a Reading on the Road visit (see Outcomes 1 and 3) relates to the mobility of this population and the difficulty of reaching them.  The staff member who made the calls to these clients reached only 58 parents (36%) out of a total of 161 calls.  In addition, a mailing that included a flyer about story time and a coupon for a free book if parents visited the library was sent to WIC clients who had received library cards in years 1 and 2 of the project.  The post office returned as undeliverable 14% of the mailings sent to first-year library card receivers and 6% of those who received a card in the second year.

d.   The in-person interviews of health clinic staff (see Outcome 7) asked respondents to give examples of (a) how the program has affected their interaction with their patients about literacy and (b) ways the program is impacting parents’ and children’s reading behavior.  The following are examples of answers to those questions.  

• Lots of my parents don’t read or write.  I am teaching them to talk about the picture to their kids—that way they will look at books with their kids.
• I walk in on parents who have children sitting on their lap pointing to the pictures.
• Kids are enthusiastic about receiving a book as soon as they arrive.  Often, the parent is reading the book while they wait for the doctor.  Books are fun to look at, illustrations excellent, as well as text.
• Allows me to promise the kids a special new book if they allow an exam, or we use the book during the exam and it helps me judge their development.
• I tell moms (and dads) of babies about how they can start to read to them at an early age.
• The smaller ones reach out for books, point at things in books, are happy to get a book—recognize books are fun.
• The children run to our supply of books in the rooms and want to look at them and read with their parents.  Sometimes we need to pry them away (so to speak) from the books.
• Most parents didn’t know that they should start reading to their babies so young.  When told to start as babies, they do and are often pleasantly surprised at their babies’ delight in the books and the interaction.
• Discourages TV watching, encourages visiting the library.
• I believe more children are being read to at home because of the program.
• Parents tell us that not only do they read to their kids, but now the siblings are reading to each other!
• Every day, when a new family who is unfamiliar with our Reach Out and Read program has a scheduled well-physical, a ray of light is shed over parent(s) and child as they see the joy in the child’s face while reading a new book.

3.  Summary and Conclusions

This evaluation shows that the early literacy intervention done by the San Mateo County Library during visits to WIC sites and visits to health clinics are helping to spread knowledge about the importance of early literacy among low-income parents, some of whom are barely literate themselves, and most of whom are not in the habit of reading to their children.  The fact that the programs are offered in places where parents receive services for their children means that the message is conveyed in a supportive setting.  The free books distributed by both programs may be the first ones in the home, and having the books encourages parents to read more to their children and use the library services they learn about for the first time.  The fact that 513 parents signed up for library cards during WIC visits in the two years of the Reading on the Road program and that 29% of them used their cards this year (up from 9% last year) suggests that the message is getting through—reading is fun, accessible, and healthy for both children and parents.

The programs are meeting their outcome goals, as detailed in an earlier section.  Parents who received a library card from Reading on the Road are more aware of the importance of early literacy and more knowledgeable about library services.  They report spending more time reading to their children at home, and they have increased the use of their library cards.  Staff members of WIC sites visited by Reading on the Road have learned new skills to build literacy in young children and are very satisfied with the services offered by the program.  Health care staff in clinics are very enthusiastic about that program’s ability to encourage parents to begin reading to their children at an early age, and they are very appreciative of the free books they can give away to children during well-child visits.  The number of books in children’s homes has definitely increased with the over 16,000 books given away during this year alone.

The answers to survey questions and the records of books distributed and library cards used provide statistical evidence of the programs’ impact on parents and children and on the staff who serve them at WIC sites and health clinics.  It is the comments made on surveys or in interviews, however, that bring the results to life.  WIC staff and health clinic staff firmly believe the program is helping parents to understand the importance of reading to their children and to bond with their children through the sharing of books.  They appreciate the way the program is run and believe it is impacting their clients in many positive ways.  Though not asked this question, it is clear from their comments that they want the program to continue providing early literacy services to this very vulnerable population.