INTRODUCTION

The boom in information and communication technologies (ICTs) has been accompanied by breakthroughs in information management and new perspectives and demands on knowledge management. In Cuba, the economic crisis that began in the early 1990s (known as the “Special Period”) resulted in a sharp decline in availability of Cuban and international printed literature. At the time, ICTs provided a relatively economical alternative for obtaining current information. In this context, Cuba’s national health care and telecommunications network and portal – INFOMED – was created, offering a solution to the concrete problem at hand and a strategy for the future.[1,2]

INFOMED’s strategy included the creation of products and services compatible with the new information access model that the Internet had begun to develop. Several projects were launched to enable access to information resources. Courses and interactive spaces were generated to facilitate health professionals’ continuing education and individual study, as well as communication among themselves and their centers, and with colleagues and institutions in other sectors and abroad.[3,4,5]

INFOMED’s creation in 1992 responded to a Ministry of Public Health (MINSAP) strategy and, since then, the Ministry has continued to strengthen the network.[6] As a result, INFOMED has gradually established connectivity with all Cuban provinces, prioritizing creation of national capabilities to appropriate these technologies for social use in places such as libraries, laboratories, and network access points (“cafes”). Emphasis was placed not only on the use of the technologies, but also on empowering people to use them for individual and collective problem-solving. [1,7]

The underlying principle was that if ICTs were to have a transforming effect, then their use should be based on an assessment of the needs to be met accompanied by concrete goals, rather than on the technology in and of itself. Thus, INFOMED’s first priority was training of human resources and mobilization of health workers’ creative potential to find solutions to the complex problems they were confronting on a daily basis.[6,8]

The fact that health care in Cuba is organized into a single, integrated system facilitates adoption of national strategies to carry out policy, and implementation through concrete actions nationwide. One of these strategies is primary health care, in which neighborhood-based family doctors play an essential role. However, because of their large numbers and geographic dispersion, these physicians run the risk of professional stagnation unless they have a system of continuing education and access to information for professional development. This dilemma is shared by the many other professionals and technicians whose work is key to quality health care delivery at the primary level and to the health of their communities, generating further concern and reason for action.[1]

ICTs offer benefits for primary health care since they address the problems of distance and vertical workflow, through the opportunities facilitated by networking and multiple links among the various actors. The introduction of these technological advances then begins to modify scenarios and relationships, which can be strengthened to contribute to the aims of the health sector and the decision-making process.

Hence, the Program to Introduce Information and Communication Technologies in Primary Health Care (PICT-PHC) was established. Networked computers were installed in all community polyclinic libraries as health professionals’ local access points to information resources available via INFOMED. These resources include: the Virtual Health Library, providing a number of information sources and services reflecting the health system’s priorities; the Virtual Health University, a collaborative learning environment which contributes to continuing education with courses and other academic modalities; bibliographic databases; and collections of full-text articles, books and other documents – to cite a few examples.

INFOMED is a network used by a set of individuals and institutions to integrate their actions and manage information and knowledge aimed at achieving the health goals of a given community or territory.[1,2]

In 2001, the PICT-PHC was piloted in 6 Havana polyclinics. By late 2003, the Program was extended to 81 polyclinics nationwide; and by the end of 2004, it was in place in all 444 polyclinics in the country at the time.

The PICT-PHC’s aim is to fulfill the policy objectives of making information more readily available to health professionals and providing them with a fundamental tool for knowledge management. INFOMED’s directors requested the National School of Public Health conduct an assessment of phase one of the PICTPHC in Havana. The study was intended to determine whether the policy objectives were being fulfilled, and what factors had thus far facilitated or hindered the program’s implementation in the selected universe, as a basis for applying appropriate corrective measures.[9]

The objective of this paper is to evaluate the factors in the Program structure, implementation process, and outcomes which have had the greatest impact on effectiveness of its phase one implementation in Havana.

MATERIALS AND METHODS

A descriptive, cross-sectional study was conducted. The universe consisted of the 16 Havana polyclinics where the PICT-PHC had been in place for no less than 6 months as of May 2003. Surveyed were all polyclinic librarians (41) and all persons who had used library services at any time in a one-month period during our fieldwork period. The study paid particular attention to information professionals and technicians and their competencies, given their role in providing services and orientation to library users.

The research was based on the ethical principles of respect for voluntary participation, impartiality, and data compartmentalization. Global results of the program were presented, rather than those of institutions, groups or individuals. Data revealing lack of skills, knowledge or interest on the part of study participants were kept confidential in the custody of researchers.

The research methods included the following four steps:

1. Determination of Dimensions, Criteria and Standards

Work began with a review of relevant literature and Program design documents, to identify the Program’s objectives and its theoretical and methodological bases. A systemic approach and the three-component conceptual basis of Donabedian’s model – structure, process, and outcome – were applied.[10,11]

Dimensions, criteria (each expressed in the form of an indicator) and standards were determined through consultations with experts.[12] Work with these specialists was organized from the general to the specific: dimensions were defined first; then the criteria for each dimension and their corresponding indicators; and, lastly, the standards applicable to each criterion. Standards established by the experts were high, given the Program directors’ express interest in identifying those factors hampering successful implementation of the Program from its inception. For a dimension to be considered fulfilled, at least 70% of its criteria had to meet the established standard.

Experts were chosen chiefly based on the importance and relevance of their research. They were selected from among: scientific and technical information specialists with at least 10 years’ experience at the National Medical Library or in polyclinic libraries; program directors at INFOMED; National School of Public Health researchers with experience in program evaluation; and ICT users with more than 10 years as professionals in the national health system.

This study presents the dimensions with the greatest impact on the effective use of ICTs in polyclinics: for the area of structure, librarians’ competencies; for process, training received by librarians, training received by users, and program monitoring and follow-up; and for outcomes, exploitation of ICTs.

The “librarians’ competencies” dimension was designed to assess the main skills required for ICT use (Table 1).

The “training received by librarians” dimension was designed to determine whether the formal training activities set forth in the Program were carried out, and whether librarians acquired the necessary skills as a result (Table 2)

Table 1: Criteria and Standards for “Librarians’ Competencies” Dimension

Table 2: Criteria and Standards for “Training Received by Librarians” Dimension

The “training received by users” dimension was designed to assess the process of ICT skills development among polyclinic workers (physicians, dentists, university-level nurses and other university graduates, and technical-level nurses) through formal training (Table 3).

This dimension relates to the perceptions of users about course contents.

Table 3: Criteria and Standards for “Training Received by Users” Dimension

The “program monitoring and follow-up” dimension was used to assess polyclinic managers’ activities to systematically monitor Program implementation (Table 4).

The “exploitation of ICTs” dimension was intended to evaluate utilization of ICT installed capacities, to use the information and services available through INFOMED (Table 5).

Table 4: Criteria and Standards for the “Program Monitoring and Follow-up” Dimension

Table 5: Criteria and Standards for “Exploitation of ICTs” Dimension

Note: No. of users refers to those who reported using ICTs for the specific purpose expressed by each criterion.

2. Design and Validation of Data Collecting Instruments

Six instruments, and their respective instructions, were created for data collection. These included: a guide for polyclinic visits, a guide for interviews with library directors, and a guide for evaluating librarians’ IT skills; plus one questionnaire for librarians and another for users, and a data collection model (from the polyclinics’ user records).

Instrument content validation was performed through consultations with experts, and feasibility was determined with a pilot study at the Heroes del Moncada Teaching Polyclinic in the Plaza de la Revolución municipality.

3. Field Work

The instruments were applied by the authors (except for the users survey, which was applied by librarians) at the 16 polyclinics included in the study. Prior to this, a workshop was held for library directors, concerning the objectives, instruments, and ethical principles of the study. The fieldwork phase took place between May and October, 2003.

4. Data Processing and Analysis

The data used to calculate indicators were processed using descriptive statistical methods, mainly percentages and mean values. [13] The results obtained were compared with the previously established standards, which facilitated a comprehensive assessment of the Program.

RESULTS

Of the 16 polyclinics studied, it was found that 8 served as Municipal Information Centers, and that undergraduate courses were offered in all polyclinics. During fieldwork, 32 of the 41 librarians were interviewed, and the 544 users who participated in the study were surveyed. All library directors were interviewed, who provided information on the training received by their staff.

Assessment of program structure: Three criteria (50%) of the “librarians’ competencies” dimension were met (Table 6). The study revealed lack of competencies among librarians in vital aspects of information management such as searches in bibliographic databases and the Virtual Health Library.

Table 6: “Librarians’ Competencies” Dimension

The fact that the ‘e-mail use’ criterion standard was not met may be due to the economic limitations affecting the Program in its early stages, in which only one e-mail account was available at each polyclinic library, usually managed by the library director, without access by other librarians. A good number of the librarians were Scientific and Technical Information (STI) technicians, and they attained good results in this dimension, and general training levels were satisfactory among the personnel studied (Table 7).

Table 7: Librarian Competencies by Results and Scientific and Technical Information (STI) Specialization

*Three-year program after 9th grade †Based on number of people evaluated

Assessment of program process: In the “training received by librarians” dimension, 5 (71.4%) criteria standards were met. In all polyclinics studied, at least one librarian had been trained under the Program. The ‘Virtual Health Library searching’ and ‘bibliographic database searching’ criteria standards were not met. Both are essential for providing quality library services. It is noteworthy that, overall, the training provided to librarians met the established standard. However, the above two subjects presented difficulties when course contents were analyzed more in-depth. A majority of library directors expressed the need to focus training on specialized information searches and to provide continuous training opportunities for librarians, to help them meet the new demands of their work (Table 8).

Table 8: “Training Received by Librarians” Dimension

Table 9 shows the training received by librarians, by level of specialization.

Table 9: Training Received by Librarians by Scientific and Technical Information (STI) Specialization

* three-year program after 9th grade.

Table 10 shows the relationship between the number of librarians trained and their IT skill level. The Program did not include training in computer use, but this was nonetheless explored in view of its relevance. Of the librarians assessed, 93.8% did well according to this criterion, followed by INFOMED navigation at 81.3%.

Only formal courses for developing ICT skills were considered for the “training received by users” dimension. Medical schools had offered courses for polyclinic physicians, dentists, university-level nurses and other university-level graduates, as well as technical- level nurses, explaining the positive evaluation for the ‘users trained’ criterion (Table 11). Users’ perceptions and suggestions were surveyed concerning content of courses and their balance between theory and practice.

Table 10: Training Received by Librarians by IT Skills

In this dimension, the ‘users trained’ criterion standard was met. Users reported that INFOMED surfing and e-mail use were the two topics most commonly covered in the courses. In the surveys conducted, 36 of the 290 users trained made suggestions on some aspects of the courses, such as their duration, the requirement for them to be completed, and the time devoted to practice activities, among others.

Table 11. “Training Received by Users” Dimension

Concerning the dimension “program monitoring and follow-up” by polyclinic management, a distinction was made between criteria related to implementation and those related to results (Table 12). For this dimension, 3 of 4 criteria standards were not met, with positive results only in the area of statistical reporting. However, almost none of this information was submitted at the polyclinic level, and it should be stressed that reports still did not reflect all the information required for program evaluation.

Assessment of program outcomes: As to the “exploitation of ICTs” dimension (Table 13), only two criteria were met (33%). ‘Information searches’ scored best, while criteria on the use of ICTs for participation in teaching activities, virtual conferences and communities, scientific exchanges, and working groups scored very poorly against the established standards.

Table 12: “Program Monitoring and Follow-Up” Dimension

Table 13: “Exploitation of ICTs” Dimension

Libraries provide services to polyclinic workers and to members of the local community. Users also include health workers at other institutions, who use the polyclinic library nearest home. Students are the most frequent library users outside the health sector. Table 14 shows ICT use by these three groups at the polyclinics studied.

DISCUSSION

Primary Health Care (PHC) is a fundamental pillar of Cuba’s health system, emphasizing increased ability to solve patients’ health problems at this level, as well as increased coverage and accessibility to health care services. To this end, primary health care centers must improve the quality of their services by enhancing the competencies and performance of their professionals and staff in general, with an approach centered on community work, intersectoriality and evidence-based decision-making. Access to relevant information sources is therefore paramount.

The importance of ICTs for primary health care and health systems has been widely recognized in the international scientific literature.[14,15] The Program to Introduce Information and Communication Technologies in Primary Health Care (PICT-PHC), aims to make information more readily available for end users, providing them access to current resources, and facilitating scientific exchange and continuing education.

The overall results of the dimensions studied show a consistency, the findings corroborating each other. The “librarians’ competencies” dimension did not fulfill the established standard, indicating gaps in some skills that are essential to their work, such as searching the Virtual Health Library and bibliographic databases. This concurs with the results of the survey on the training received and also with suggestions made by teachers, who requested more depth in topics such as these, where difficulties had been noted.

The assessment of the Program’s first phase showed that over 50% of library users had received training on the use of ICTs through continuing professional development courses provided at medical schools. However, the courses did not cover all users, and their duration did not satisfy all needs. The insufficient exploitation of ICTs may point to the need for expanding courses contents and increasing the amount and duration of their practical activities.

Table 14: Users, Grouped by ICT Use and User’s Link with Polyclinic

*With peers, experts or professors †Participation in forums, discussion lists, chats or group debates ‡Case studies, questions to experts, distance-learning courses

In developing the Program, ways had not been foreseen to meet the continuing education needs of librarians who were not university graduates. Our results suggest that short courses do not adequately fulfill these needs. A certificate course of longer duration was in place, but only for university graduates, covering health information management (including face-to-face and distance-learning modalities).

As we expected, of all the dimensions included, those related to users presented the most difficulties, since they involve a greater breadth of individuals. Consequently, training requires more effort and resources. The literature reviewed reports that users normally require a longer time to acquire ICT skills.[16,17] These studies describe this phenomenon as a natural process, indicating that the most difficult challenge is to develop an ICT culture among users – providing assistance, when necessary, to build satisfaction and confidence.[17,18]

It has also been reported that “initially, users tend to have a rather passive attitude, and will just access and download information on the Internet as if they were using an on-line health library. But as they get more and more used to surfing the Web, they will move on to accessing a larger number of sites and be able to understand their consistencies and contradictions. Then, they will start looking for answers to their questions and for further opportunities to apply their newly acquired skills.”[17]

The Program’s monitoring and follow-up was not well documented. It is clearly essential to systematically assess polyclinic use of ICTs, but this dimension was unfulfilled. Such assessments should link ICT use with the information required for decisionmaking and knowledge management in the search for solutions to health problems in the local community.

The difficulties with program monitoring also suggest that the librarians should play a more active role in knowledge and information management in the polyclinics, participating in health problem solutions. At the same time, to achieve the necessary synergy, polyclinic management should be empowered to involve librarians in the identification of information needs for research and, above all, for community-based health situation analyses.

Scores for the “exploitation of ICTs” dimension were far from fulfilling the standards established, pointing to a significant gap that Program directors will have to address. One element that may contribute to this problem is the lack of indepth training in such areas as use of ICT tools for collaborative work and networked learning (courses for users as well as providers).

While the decision to implement the Program was made only after at least part of the resources were available – in this case, the material resources – the Program’s strategic development depends upon sustainability and improvement of the resources dedicated to it. In turn, this will be possible to the extent that leaders, managers, and users develop a culture of ICT use, interpreting information and communication technologies as key tools for their work.

CONCLUSION

The significance of Primary Health Care (PHC) to the national health system is unquestionable; hence, any improvement in the quality of its services and performance can have a positive impact on the system as a whole. ICT use is fundamental at this level, since it constitutes a tool to empower primary care professionals and technicians, leading to improvements in their health services, problem-solving capacities, and efficiency.

The Program to Introduce Information and Communication Technologies in Primary Health Care (PICT-PHC) was designed to make information resources more readily available for health professionals and technicians. It has had a positive impact during the first phase of implementation in Havana, making an innovative contribution to provision of access to end users, who recognize its importance in facilitating current information and tools for scientific exchange and continuing education.

Difficulties were found in some competencies fundamental to the librarians’ work; and Program monitoring was aimed more at process than at outcomes. This has limited the involvement of librarians in information and knowledge management at the polyclinic level and their active participation in important processes such as health situation analyses, teaching and learning activities, and support for health care, training, and research staff.

The problems found with user training are expected during this phase of the Program, and are expressed in deficient use of ICT availability, resulting in generally insufficient exploitation. We recommend expansion of training activities.

Taken globally, the results of our study point to the need for Program directors to place greater emphasis on ICT training for librarians and other polyclinic users.

THE AUTHORS

Susana B. Llanusa Ruiz (Corresponding Author: susy@infomed.sld.cu), electrical engineer, specialist in digital systems. Researcher and associate professor, National School of Public Health, Havana, Cuba.

Nereida Rojo Pérez, full professor and Deputy Director for Science and Technology, National School of Public Health, Havana, Cuba.

Magali Caraballoso Hernández, specialist in epidemiology. Associate researcher, consulting professor, National School of Public Health, Havana, Cuba.

Julia S. Pérez Piñero, specialist in biostatistics. Associate professor, National School of Public Health, Havana, Cuba.

ACKNOWLEDGMENTS

We thank Roberto Capote Mir, MD, for his encouragement during the course of this study, the INFOMED directors for their invaluable assistance, and the librarians for their interest and cooperation.

MEDICC Review, Spring 2008, Vol 10, No 2