Friday, October 19, 2012

JMIR--eHealth Program to Empower Patients in Returning to Normal ...

'); } //]]>?


Step 1: Needs Assessment

In needs assessment, the discrepancy between the current and the desired situation in a given group of people is studied. The needs assessment was structured by the Precede-Proceed model (PRECEDE: predisposing, reinforcing, and enabling constructs in educational diagnosis and evaluation; PROCEED: policy, regulatory, and organizational constructs in educational and environmental development), which analyzes and correlates quality of life, health, behavior, and environmental factors in a certain population [50]. The current situation has shown a large discrepancy between expected duration of physical recovery and actual RTW after gynecological surgery (even laparoscopic), whereas there is strong evidence that long periods of sick leave can result in poorer general health, increased risk of mental health problems and work disability, and induces unnecessary costs for society [2,8] The most frequently performed gynecological surgical procedures with a considerable postoperative effect on recovery and RTW (accounting for more than 17,500 procedures in the Netherlands per year) are hysterectomy (abdominal, vaginal, and laparoscopic) or laparoscopic adnexal surgery on benign indication [51]. Because approximately 67% of women aged 25-65 are in the workforce, these large numbers of surgical procedures have a great impact on absenteeism [52]. Therefore, patients who underwent these types of surgical procedures were chosen as the target group for this intervention.

To clarify and find possible explanations for prolonged sick leave, a literature search in PubMed regarding behavioral and environmental conditions of prolonged sick leave and delayed RTW in the target group was performed.

The focus group technique was considered the most suitable supplement to the literature search for identification of patient?s needs, attitudes, and beliefs regarding postoperative recovery and resumption of work. In addition to supplementing the results of the literature search, it was assumed that focus group discussions would align the results to the Dutch context and give more insight into specific content requirements of the prospective eHealth intervention that could be used during the development process. The participatory technique of focus group discussions is widely used and scientifically accepted to gain insight into public views and needs through group interaction [53,54].

Participants for the focus group discussions were recruited from the patient files of the VU University Medical Center, an academic hospital in the Netherlands. To mirror the intended target group, inclusion criteria for participation in the focus group discussions were age 18-65, a history of a laparoscopic adnexal surgery and/or hysterectomy on benign indication in 2008, and a job (paid or unpaid) of at least 8 hours per week. To create homogeneity within the focus groups but heterogeneity among the groups, the patients were recruited by means of purpose sampling into 3 groups: fast RTW, intermediate RTW, and delayed RTW. All of the participants had already returned to work after surgery (range 1-36 weeks).

The aim of the focus group discussions was to identify patients? needs regarding perioperative care and counseling in resuming normal and work activities. In addition, patients were specifically asked for the important requirements of a useful eHealth intervention. The identification of patients? needs and requirements occurred in 3 different steps:

1. Identifying and prioritizing patients? perceived shortcomings in and difficulties with received perioperative care and counseling in resumption of normal and work activities.

2. Inventory of possible solutions and improvements to overcome these shortcomings and difficulties, starting with the highest prioritized bottlenecks.

3. Brainstorming about favorable content, requirements, and specific tools that should be incorporated into an eHealth intervention that aims to empower patients during the perioperative period and resumption of work activities.

The focus group discussions were all recorded and transferred into verbatim transcripts that were analyzed by open coding using the ATLAS.ti software [55].

A detailed process evaluation of the focus group discussions will be published in a separate paper [Pittens et al, unpublished data, 2012]. The study design and procedures of the focus group discussions were approved by the Medical Ethics Committee of the VU University Medical Center (2009/42, February 9, 2009). Participants signed a privacy agreement to declare voluntary participation, to give permission for processing the information for the development of an intervention (such as an eHealth intervention), and to exclude transmittal of information to others.

As a starting point for the development of the intervention, the products of this first step were the main behavioral and environmental conditions of the chosen target group contributing to prolonged sick leave.

Step 2: Matrices

The purpose of this step was to transform the identified behaviors and environmental conditions causing prolonged sick leave into behaviors and conditions that prevent a prolonged sick leave. To achieve this, performance objectives were formulated. Performance objectives describe in detail patients? behavioral and environmental outcomes that are necessary for patients to reach the formulated behavior objective of ?timely RTW.?

To select a suitable theoretical framework to reach the performance objectives, a literature search regarding main determinants of recovery and RTW was performed in PubMed. To elucidate, a suitable theoretical framework provides appropriate determinants that could be influenced to reach the behavior objective. Based on this framework, the performance objectives of the target group were elaborated into matrices with change objectives, explaining how patients and their environment will change as a result of the eHealth intervention to reach the behavior objective.

Step 3: Theory-based Methods and Practical Strategies

In this step, theoretical methods and practical strategies to address the change objectives were searched for and applied. Research has shown that the effectiveness of interventions to change behavior can be increased by the use of theory-based methods [56]. A theory-based method is a method derived from theory and research that describes a process that influences changes in determinants of behavior and environmental conditions. A practical strategy is a technique for the application of the theory-based method in ways that fit the target group and the context in which the eHealth intervention will be applied. The required theoretical framework, theory-based methods, and translation into practical strategies were determined based on the book that describes the IM approach [42], a literature search in the PubMed database, the focus group discussions, and a brainstorm session of the researchers.

Step 4: Program Plan and Design of the Intervention

During this step, information obtained in previous steps was translated into specific tailored tools and information to empower gynecological patients by the eHealth intervention. Furthermore, to obtain evidence-based information and instruments necessary to fulfill patients? needs, additional research was performed.

To verify if the eHealth intervention matched with the main target group and fitted the expectations of gynecologists, family physicians, and occupational physicians, the first concept version was evaluated by focus group participants (n = 21), physicians (n = 22), eHealth specialists (n = 3), and a representative of a patient organization (n = 1) through questionnaires. The eHealth intervention was scored on 8 main areas used to describe how the intervention functions, empowers, and can be modified to provide the best behavior change to obtain timely RTW and prevent work disability. The 8 areas included: appearance, behavior prescriptions, burdens of using the website, content, delivery, message, participation, and assessment and tailoring. Ritterband et al [57] describe these areas in detail. This model is meant to ground Internet intervention research within a scientific framework. The 8 different areas were covered in the evaluation questionnaires with 23 unique open- and close-ended questions (Appendices 4 and 5). In addition, participants were also encouraged to propose recommendations. The results of the evaluation were used to optimize the design and usability of the eHealth intervention, which resulted in the final version.

Step 5: Design of an Implementation Plan

The focus of Step 5 is adoption of the intervention by the patients and relevant stakeholders, and the development of an implementation plan. With the input of patients and stakeholders during previous steps, the researchers identified facilitating factors and barriers regarding adoption and implementation of the eHealth intervention. With this information, an implementation plan to enable an extensive evaluation of the intervention was developed and an appropriate linkage system for future implementation was composed.

Step 6: Design of an Evaluation Plan

During Step 6, the main objective of this study (ie, to develop a feasible and generally accepted eHealth intervention that empowers gynecological patients during the perioperative period into returning to normal activities and work, to obtain timely RTW, and prevent work disability) was used to compose an evaluation plan. Although the eHealth intervention was based on both theory and evidence and was developed in collaboration with the main target group and relevant stakeholders, its adoption, barriers for usage, and implementation possibilities still had to be evaluated in daily practice. In addition, the effectiveness of this eHealth intervention regarding a timely RTW to prevent work disability had to be investigated. Therefore, the project group approached 7 gynecology practices (1 university-based and 6 hospital-based) about participation in the evaluation of this intervention through implementation of the eHealth intervention as a supplement to the standard perioperative care given at their hospital. In addition, the project group formulated inclusion and exclusion criteria for patients to participate in the study and developed appropriate outcome measures to evaluate the intervention?s effectiveness, adoption, usage, and implementation. Furthermore, a logistic plan to recruit patients and involve participating health care providers was developed.


Step 1: Needs Assessment

Literature

The literature search revealed that most women extend their sick leave beyond the recommended period on their own initiative [2]. Patients with delays in RTW reported pain/discomfort, feelings of fear, and infections as delaying factors [1]. Those who reported multiple delaying factors reported a variety of combinations that included feelings of fear, anxiety, depression, and differences in employer expectations [1]. Recovery and RTW time is shorter when the patient receives clear and few restrictions that are not too overly cautious at discharge, when the patient has been provided with RTW advice, or when the patient feels urgency to RTW [1,3,58]. Other important environmental conditions for prolonged sick leave and RTW of patients appeared to be the substantial variation in convalescence recommendations given by gynecologists, family physicians, and occupational physicians [58,59]. Their recommendations are not related to the most successful return to normal and RTW activities or the risks of complications [58]. In addition, a lack of clarity regarding absence duration can provide an obstacle for employers and employees who are keen and willing to establish earlier rehabilitation programs, but would not wish to go against the advice of health care providers [59].

Focus Group Discussions

Out of 105 invited patients, 38 met our inclusion criteria and were willing to participate in the focus group discussions. On the basis of availability on the selected dates for the focus group discussions, 31 patients were assigned to 3 focus groups. Of these patients, 21 were present at the meetings and participated in the focus group discussions (7 patients per meeting). A process evaluation of the focus group discussions will be published in detail elsewhere [Pittens et al, unpublished data, 2012].

Starting with the first aim of the focus group discussions, the most important reported shortcomings and difficulties of currently provided perioperative gynecological and reintegration care were (in random order):

1. Insufficient or no information about the surgical procedure itself, the logistical process in the hospital from admission to discharge, detailed resumption of work activities after the surgical procedure, and the possible consequences of the surgery (physical and mental).

2. Inconsistency of convalescence recommendations given by gynecologists, family physicians, and occupational physicians.

3. Lack of written instructions on resumption of work activities, tailored to individual conditions and work, and consequently insufficient information and instructions given to relatives.

4. Insecurity with respect to physical or mental postoperative symptoms, complications, or delayed recovery. What to do and whom to contact?

5. Poor communication among gynecologists, family physicians, and occupational physicians resulting in inadequate transfer of information about the procedure and one another?s treatments.

6. Limited or inadequate guidance by occupational physicians because of a lack of knowledge about different types of surgery and corresponding recovery times. Patients reported experiences of occupational physicians forcing the patient to RTW too early or slowing down the RTW process.

7. Difficulties with work reintegration because of insufficient involvement and understanding of the employee/employer during the perioperative and reintegration period.

8. Inability of patients to discuss the perioperative period and reintegration process at work (with employer and colleagues).

9. Lack of a reintegration plan before the surgery.

10. Few opportunities to contact other patients to exchange experiences.

In general, patients mentioned that when they were unsatisfied with the information or counseling given by their doctors and nurses, they asked family and friends who had undergone surgery about their experiences. However, this led to unrealistic expectations because of different types of surgical procedures and techniques, and the fact that recovery is affected by individual conditions and circumstances.

In the second part of the focus group discussions, the patients brought up many possible solutions and improvements to overcome the mentioned shortcomings and difficulties that were processed into performance objectives during Step 2.

Requirements, content, and specific tools that should be incorporated into an eHealth intervention to improve empowerment during the perioperative period may be summarized as follows:

1. Reliable detailed and personalized information about mentioned shortcomings and difficulties in the information supply. Pictures and videos were considered an accessible supplement to transfer this information.

2. Tools for communication with other patients, employers, gynecologists, occupational physicians, and family physicians.

3. Functionalities to develop a personalized reintegration plan.

With the results of the literature search and focus group discussions, the project group concluded that the main determinants of patients? behavior regarding prolonged sick leave are: (1) inadequate knowledge of important information about the surgery, recovery, and RTW; (2) tendency to extend their sick leave beyond the recommended period; (3) insecurity about postoperative symptoms, complications, and delayed recovery without knowing where to receive appropriate help; (4) lack of skills to compose a work-reintegration plan and to identify possible barriers for RTW; and (5) lack of knowledge about the opportunity to develop and discuss a work reintegration plan before surgery with the employer and an occupational physician. In addition, important environmental conditions of patients? behavior are considered to be: (1) inconsistency and lack of clarity in convalescence recommendations given by gynecologists, family physicians, and occupational physicians; (2) lack of communication among gynecologists, family physicians, and occupational physicians; (3) lack of clarity from health care providers about who to contact in case of postoperative complaints; (4) lack of initiative of the employer and/or occupational physician to develop and discuss a work-reintegration plan before surgery with the employee; and (5) lack of involvement of the employer and occupational physician during the perioperative and reintegration period.

Step 2: Matrices

In total, 12 performance objectives derived from the main behavior objective were formulated (see Table 1).

'); } //]]>?


[view this table]
Table 1. Performance objectives to empower gynecological patients during the perioperative period and return to normal activities and work to obtain timely RTW and prevent work disability.

In addition to performance objectives for patients, there were also performance objectives formulated for gynecologists, family physicians, occupational physicians, and employers (Table 1). Nevertheless, the primary focus during the next steps of the IM protocol (the development of the eHealth intervention) will be on patients. Ideally, for each group (patients, gynecologists, family physicians, occupational physicians, and employers) an intervention should be developed specific to their needs, wishes, and behavior outcomes to minimize the risks of theory and/or program failure. However, a balance had to be found between the ideal situation and what was within reach of this study. Secondly, the performance objectives of gynecologists, family physicians, occupational physicians, and employers could be considered external determinants of patients? behavior. These determinants can either be influenced by the patients or the patients can learn these skills through the intervention and how to handle them adequately. Finally, the performance objectives of gynecologists, family physicians, occupational physicians, and employers are relatively simple objectives to reach. The researchers are convinced that the main part of these objectives can be reached through making agreements with gynecologists, family physicians, occupational physicians, and employers and by involving them in the evaluation and implementation plan (IM Steps 4-6) without specifying determinants of their behavior and applying specific theoretical methods and strategies for them.

The literature search showed that the main determinants of recovery and return to normal activities and work (in addition to the physical condition of the patient, level of invasiveness of the surgical procedures, and related complications) are the patients? attitude, social influence, and self-efficacy [60-65]. In addition, skills, barriers, and facilitators are important factors that influence RTW [66-68]. For these reasons, the Attitude-Social influence-self-Efficacy (ASE) model [69,70], adapted for recovery and return to normal and work activities, was used to affect behavior of patients (see Appendix 1). The ASE model is comparable to the theory of planned behavior [71], which describes the relation between attitude and behavior. The modified ASE model describes that the behavior of a patient after surgery regarding recovery and return to normal and work activities is determined by attitudinal beliefs, social influence, and self-efficacy beliefs, and is influenced by skills, barriers, and resources. The ASE model was used to create matrices with change objectives. To fill out the matrices, available literature regarding the performance objectives and determinants was studied together with the results of the needs assessment and expertise of the project group. Appendix 2 presents an example of the change objective ?Patients develop a work-reintegration plan.?

Step 3: Theory-based Methods and Practical Strategies

Numerous practical methods and suitable strategies to affect all formulated determinants were identified and used for the development of tools and materials of the eHealth intervention. Appendix 3 presents some examples of these methods with preconditions for the method necessary for it to succeed [72] and final tool/materials of the eHealth intervention. References and footnotes explain the source and development process of each method, strategy, and tools/materials.

Step 4: Program Plan/Design of the Intervention

With the knowledge obtained in the previous steps of the IM protocol, the project group convened at several meetings to invent various appropriate tools for the eHealth intervention. A website producer specializing in eHealth interventions and a screenwriter were consulted at some of the meetings. In addition, an experienced gynecologist outside the project team was consulted to judge the medical content of one of the tools.

In close collaboration with the website producer, the eHealth intervention was developed with MODX, an open-source hypertext preprocessor (PHP) Web application framework with a capable built-in content management system (CMS). The Internet address of the eHealth intervention is ?http://www.ikherstel.nl/www.ikherstel.nl,? which means, ?I am recovering? (Appendices 6-10) [73]. The eHealth intervention was developed with special attention to colors, layout, navigation, and readability to create confidence and user-friendliness. For the patient, it consists of two main sections: an Action List to assist in resumption of activities and a central home page. Gynecologists, family physicians, and occupational physicians have access to a different section. Table 2 presents an overview of the tools in the eHealth intervention. For some tools, additional information about the development and functioning is described subsequently.

'); } //]]>?


Action List

When a patient logs onto the eHealth intervention, she will be immediately directed to the Action List. This list consists of different tools developed to target specific determinants, aimed at encouraging return to work activities, coaching patients in case of uncertainties, answering possible questions, prevention of common pitfalls, and improving communication among the patient, care providers, and the employer. An algorithm based on the date of surgery determines the priority in which the different actions should be performed to improve the recovery process. Tools of the action list are:

Composition of a Work-Reintegration Plan

By using this tool, the patient is able to select activities that are required to fulfill her work activities and at what level (eg, lifting 5 kilograms or walking 1 hour.). Consequently, on the basis of the operation date and how the surgery went (input of gynecologist), the eHealth intervention provides the patient with tailored advice about when these activities are thought to be medically safe to resume. The recommendations are based on the results of a modified Delphi study, in which an expert panel of gynecologists, family physicians, and occupational physicians developed detailed multidisciplinary convalescence recommendations for resumption of work activities after hysterectomy and/or laparoscopic adnexal surgery [74]. Moreover, this part of the eHealth intervention provides an overview of potential bottlenecks for reintegration and motivates patients to consider if work adaptations are required temporarily. A printout can be made to discuss the advice with the employer and/or occupational physician to develop an extended reintegration plan.

Resumption of Normal Activities

This functionality guides the patient to compose a detailed tailored plan about the gradual resumption of various daily activities (eg, climbing stairs and vacuum cleaning). Recommendations are based on the results of the modified Delphi study [74]. This tool also evaluates if help is needed for tasks such as housekeeping or taking care of young children. A printout can be made to share with relatives or friends.

Evaluation of Complications

When a complication has occurred, the eHealth intervention carefully determines through a survey which symptoms require additional consultation with care providers or adaptation of the convalescence recommendations. The project group developed the survey and determined which symptoms are severe complications. If the tool is not able to provide recommendations under these circumstances, an email will be sent to inform the gynecologist of the condition of the patient in order to evaluate her symptoms and possible consequences.

Home Page
Video

Because of the influence of modeling behavior on attitude, a video was chosen as the most appropriate medium to deliver an informative message to patients and relevant stakeholders about common pitfalls during the perioperative and reintegration period. The video aims to prevent these problems by stimulating patients and employers to discuss potential problems and to develop a reintegration plan to facilitate and improve reintegration. The experiences of the patients in the focus group discussions were converted into common pitfalls for patients, employers, and health care providers during this period, and a screenwriter processed them into a script for a video showing two cases of a good and bad interaction between a patient and her environment. The screenwriter worked together closely with 3 gynecological patients to make the video geared to the patients? perception of the perioperative and reintegration period.

Recommendations for Employee and Employer

Based on the experiences of the patients in the focus groups, the researchers formulated main recommendations for patients and employers regarding a successful reintegration.

Frequently Asked Questions

Answers to questions brought up during the focus group discussions and those found as main topics in patients? brochures or in discussions of gynecological patients on the Internet were formulated by the researchers (based on the literature and clinical experience) and put into patient leaflets. An experienced gynecologist outside the project team judged all questions and answers on reliability and clarity, and suggested possible adjustments.

Glossary

Based on the literature, an explanation of the most frequently used medical terms was provided by the researchers.

Links to Other Websites

The researchers searched the Internet for the most relevant websites for gynecological patients and made a selection based on relevance, reliability, and clearness of the information.

Section Aimed at Gynecologists, Family Physicians, and Occupational Physicians
Guidelines

Multidisciplinary guidelines with well-defined convalescence recommendations after uncomplicated hysterectomy (laparoscopic supracervical, total laparoscopic/laparoscopic-assisted, vaginal, and abdominal) and laparoscopic adnexal surgery on benign indication are provided. Recommendations are based on a modified Delphi study.

Casuistry

Classic examples of indications for surgery, perioperative course, and recovery after uncomplicated hysterectomy or laparoscopic surgery were developed based on literature and clinical experience of the project group.

Background Information

Elucidation of different types of hysterectomy and laparoscopic adnexal surgery concerning surgical technique, level of invasiveness, and medical consequences were formulated by the researchers.

Test Phase

Fifteen patients, 11 physicians (gynecologists, family physicians, and occupational physicians), 3 eHealth specialists, and 1 representative of a patient organization completed the evaluation form regarding the demo version of the eHealth intervention. Appearance and behavior prescriptions were judged by most as pleasant, conveniently arranged, and helpful. With regard to burdens of using the eHealth intervention, almost all respondents judged the application navigation as clear and the intervention length as appropriate. However, a manual providing an overview of the different tools of the eHealth intervention was found desirable by only one of the respondents. Furthermore, two software incompatibility problems were reported. Concerning the content of the information, the way it was delivered, and the message (source and style), most of the respondents were satisfied and expected that it could empower patients, employers, and physicians. Remarks for improvement were related to supplying more detailed information about the surgery, possible psychological complications after the operation, less complicated sentences, and a more prominent place for the source of the information. Finally, participation of the patient in the treatment and the eHealth intervention?s ability to assess and tailor the recommendations to empower patients during the perioperative period and return to work activities were judged as helpful by most of the respondents. There were no suggestions for improvement of these features.

The patients indicated that their input provided during the focus group discussions was recognizably integrated into the intervention. Additionally, almost all patients confirmed that they would recommend the eHealth intervention in the current form to a friend.

Modifications Based on the Test Phase

As described previously, the respondents did not request major revisions of the eHealth intervention and only minor adjustments were proposed. Therefore, none of the original developed tools were removed from the eHealth intervention and no new functionalities were added. Following up on the suggested improvements, a manual with directions for use was added to the eHealth intervention, incompatibility problems with different kinds of software were solved, some information on the eHealth intervention was elaborated on and explained in simplified sentences, and the logo of the university hospital was added in a prominent place on the eHealth intervention. This resulted in the final eHealth intervention that was used to perform a randomized controlled trial (RCT) [75]. Screenshots of the eHealth intervention can be found in Appendices 6-10.

Step 5: Design of an Implementation Plan

In this study, anticipation of adoption and implementation started with the involvement of patients (target group) in all stages of the intervention development and evaluation. Health care providers, occupational physicians, and eHealth specialists participated in the evaluation of the intervention during IM Step 4. In addition, a committee with representatives of the Dutch medical boards of gynecologists, occupational physicians, and family physicians, and a representative of an umbrella patient organization were involved during the development of all steps of the intervention and agreed to stay involved during the final implementation steps of this intervention. Through this committee, a linkage system was created by involving the future users and implementers of the intervention from the start of the intervention development process. Furthermore, an important target of this study was to develop an eHealth intervention that could be used by patients, doctors, and employers without any support to simplify implementation. Evaluation of self-reliant use by patients and important stakeholders was evaluated positively during the test phase of Step 4.

Within the context of a RCT with the eHealth intervention (Step 6), the project group will facilitate its implementation and maintenance. In collaboration with the relevant care providers, the eHealth intervention will be offered as a supplement to standard perioperative care and will involve minimal additional time investment for the care providers. Agreements about usage of the contents of the eHealth intervention will be made with the gynecologists of participating hospitals and the family physicians and occupational physicians of participating patients. Therefore, the main purpose of this step was to create familiarity and support for the eHealth intervention and convalescence recommendations by all prospectively involved users. To reach these purposes for all of the different user groups, information letters will be distributed among patients and care providers. In addition, presentations with background information about the development of the eHealth intervention, its contents, and how to use it will be given to the gynecologists during general teaching meetings at their hospitals. Employers will become familiar with the intervention through invitation for participation by the patients (ie, employees). The eHealth intervention will be primarily used during the period of sick leave after surgery. Therefore, no agreement with the patients? employers to use the eHealth intervention during work hours will be made.

With the information gathered during the process evaluation (Step 6), in collaboration with the committee with representatives of the Dutch medical boards of gynecologists, occupational physicians, and family physicians, and the patient organization, a final implementation plan will be developed. In this plan, medical insurance companies and the Health Care Insurance Board (CVZ) will likely be involved for the final implementation of the eHealth intervention.

Step 6: Evaluation Plan

The evaluation of the eHealth intervention will be performed by a RCT, during which the eHealth intervention will be compared with usual given care at 7 participating medical centers [75]. A power calculation was performed on the primary outcome (sustainable RTW) and showed that a total participation of at least 212 patients, their health care providers, and employers should be the goal. Patients will be recruited to participate in the RCT when they are placed on a waiting list for a hysterectomy or laparoscopic adnexal surgery on benign indication in one of the 7 participating medical centers, are aged 18-65, and they work (either paid or unpaid) for at least 8 hours per week. The main exclusion criteria are malignancy, deep infiltrating endometriosis, concomitant surgical procedures, major comorbidity, sick-listed for more than 2 months, currently in a lawsuit against their employer, and not able to use the Internet or unable to understand the Dutch questionnaires. If a patient participates, the researchers will inform her family physician and occupational physician by letter about the content of the intervention, the group allocation, and what is expected of them regarding the provision of health care. Follow-up will take place approximately 26 weeks after surgery.

Patients willing to participate and who meet the inclusion criteria will be randomized to the intervention or usual care group (control group). Main outcome measures of the RCT are the effectiveness of the eHealth intervention compared to usual care with respect to RTW, general recovery, quality of life, pain intensity, and complications. Part of the RCT will be a process evaluation of the patients, their care providers, and employers in the intervention group. Main outcome measures of the process evaluation are the extent to which the eHealth intervention and convalescence recommendations are used and followed up (compliance); appreciation of the different tools of the eHealth intervention and advice; perceived effectiveness, usage, and implementation barriers; and suggestions for improvement.

The outcome measures will be obtained using questionnaires administered at baseline and at 2, 6, 12, and 26 weeks after surgery. Gynecologists will complete questionnaires 1 day after surgery for each patient and at the end of the study. Employers will be asked to evaluate the eHealth intervention 8 weeks after their employee?s surgery.

The study design and procedures of the RCT study were approved by the Medical Ethics Committee of the VU University Medical Center (#2009/218, October 22, 2009).


Main Findings

In this study, the IM protocol turned out to be a useful method to develop and tailor an eHealth intervention aimed at the empowerment of gynecological patients during the perioperative period including return to normal activities and work. By using available literature and focus group discussions, it became increasingly clear that to obtain timely RTW and prevent work disability, the intervention should target both behaviors of patients as well as environmental determinants. Performance objectives for obtaining timely RTW and prevention of work disability were formulated and matrices with change objectives, explaining how patients and their environment have to change as a result of the eHealth intervention to reach the performance objectives, were developed. Finally, based on the ASE model [69,70], theoretical methods and practical strategies, suitable tools, and materials for the eHealth intervention were developed. Most of the participating patients and stakeholders judged the intervention to be a promising eHealth tool to empower gynecological patients during the perioperative period to return to their normal activities, including work.

Strengths and Limitations

A primary strength of this study lies in the way the eHealth intervention was developed, tailored, and assessed. Both theory and evidence were combined and patients and most relevant stakeholders were involved, minimizing the risks of theory and/or program failure [72]. The frequent involvement of patients in several steps of the IM process resulted in an eHealth intervention that was specifically tailored to their needs and wishes and therefore more likely to be implemented successfully. In addition to information supply, which is the primary aim of most websites, this eHealth intervention distinguishes itself by monitoring the recovery process, giving tailor-made advice based on patients? workloads, and informing patients when additional consultation of care providers is needed. By linking patients with their gynecologists, convalescence recommendations can be adapted and insecurities regarding consequences of the complications can be solved. Connecting patients and employers facilitates a dialogue and the joint effort to compose a reintegration plan. Furthermore, this eHealth intervention is developed to be used without support and with minimal effort of care providers. Therefore, use of the intervention costs little and implementation is expected to be relatively easy. Moreover, like most eHealth interventions, an important strength is the possibility to use it at the time, place, and pace that fits the patient, care provider, and employer [38]. Finally, the combined approach of encouraging and helping patients to participate in their consultation and empowering clinicians with skills to identify and adapt to the needs of their patients is thought to produce long-term benefits for patients [21].

Main limitations concerning the needs assessment of this study include a possible selection bias; patients assigned to the focus group discussions are a selection of the patients willing to discuss their perioperative problems. Patients less willing to discuss their problems may also experience different perioperative issues. However, through purposeful sampling and by proactively approaching all relevant patients for participation in the focus group discussions, we tried to minimize this selection bias as much as possible. In addition, the influence of dominant patients who might be overly influential cannot be excluded. On the other hand, specific observations on this matter showed that this rarely occurred [Pittens et al, unpublished data, 2012]. Furthermore, these patients already underwent the surgery, whereas the intervention is designed to be used both before and after surgery. It has to be determined whether this intervention is applicable to the entire target population and whether the intervention fits the needs of patients both before and after surgery. Due to practical reasons, not all stakeholders (eg, employers and health care providers) were involved in the needs assessment and development process of this eHealth intervention. As a consequence, the intervention might be less supported by these groups. However, results of prior focus group discussions with supervisors and care providers in another comparable IM study [47] were used and some of those stakeholders were also involved in the test phase. Because this was an exclusively Dutch study directed at the Dutch health care system, a final limitation is that external validity of the eHealth intervention has to be examined before the results may be applied internationally.

Comparison with Other Studies

To our knowledge, this is the first study that tailors an eHealth intervention through the IM protocol to empower gynecological patients during the perioperative period to obtain timely RTW and prevent work disability. Therefore, comparison with other studies is limited. However, previous research showed several developmental and interventional characteristics. For example, it was demonstrated that IM is a successful method to tailor eHealth [45,76] as well as RTW [47,48] interventions. Moreover, Web-based interventions show positive effects on empowerment [25]. Furthermore, it is proven that tailoring an eHealth intervention influences usage positively (eg, time and frequency) and increases the effectiveness of the message [77,78]. In contrast to most eHealth interventions, this intervention aims at secondary and tertiary prevention. Therefore, further research is needed to determine whether the characteristics mentioned previously also apply to the present study.

Although comparable studies are lacking, the approach followed in this study?involving relevant stakeholders in the development of an eHealth intervention?is in line with an observed trend of multi-stakeholder involvement in health care in general [79,80]. Gained experiences in this study might contribute to additional insights for future initiatives on multi-stakeholder involvement in health care.

Interpretation of the Results and Policy Implications

This study shows that the IM protocol can successfully be used for the development and tailoring of an eHealth intervention for gynecological patients. The protocol led to a systematic development of the intervention, it made sure that collaboration with the main target group was realized, and both theory and evidence was used to tailor the intervention.

Furthermore, through the detailed convalescence recommendations provided by the eHealth intervention, patients will be better informed about when it is thought to be medically safe to resume daily and work activities after gynecological surgery and it will give them the possibility to arrange workplace adaptations if necessary [74]. Prospective cohort studies exploring sick leave after general surgical procedures show that return to work is primarily influenced by the expectations of the patient and their supervisors rather than physical factors or the type of surgery [1,10,81]. Therefore, it is assumed that these tailor-made convalescence recommendations will help to accelerate recovery and stimulate patients to resume activities with increasing gradations of strain, which will presumably bring about a quicker recovery and RTW and prevent work disability [82-84]. Therefore, it is expected that this eHealth intervention fulfills patients? needs and is able to empower gynecological patients during the perioperative period and return to normal activities and work [77]. However, its adoption, barriers for usage in daily practice, and implementation possibilities by patients and stakeholders still need to be evaluated more extensively in a process evaluation. Furthermore, a RCT will be needed to assess the effect of empowering gynecological patients during the perioperative period and return to normal activities and work by this eHealth intervention on work disability prevention, resumption of activities, and quality of life [75]. The results are important to assess this intervention?s true value and policy implications.

This eHealth intervention is developed for patients who underwent a hysterectomy or laparoscopic adnexal surgery. However, the strategy used to develop the intervention and the final result may also be used as a blueprint for other kinds of surgical procedures.

Conclusion

The development of an eHealth intervention according to the IM protocol to obtain timely RTW and prevent work disability by empowerment and improving communication after gynecological surgery resulted in an intervention based on both theory and evidence and involvement of patients and most stakeholders. This eHealth intervention is well accepted by patients and stakeholders and is considered to be a promising tool to obtain timely RTW and prevent work disability after gynecological surgery. Its effectiveness needs to be proven in a RCT [75].


Acknowledgments

The authors would like to thank all patients in the focus group discussions for their generous contribution of views and time. We also wish to thank all the physicians, eHealth specialists, and the representative of the patient organization who judged the eHealth intervention. In addition, we would like to thank D Stomp for the pleasant collaboration and involvement as website producer of this eHealth intervention, MMA Brood for being panel chair during the focus group discussions, LM Bolten for her dedicated correspondence with all patients, and AJ van der Ham and SC van Veen for their contribution during the focus group discussions and analysis of the transcripts. In addition, our appreciation goes to VU University?s production company, Audiovisueel centrum VU-VUmc, and director I Broekhuizen. Finally, we would like to thank Professor G Kok for his valuable recommendations regarding the description of the IM protocol.

This study was carried out with funding from ZonMw (project number 150020037), an organization for health research and development in the Netherlands. The open access publication was supported by The Netherlands Organisation for Scientific Research (NWO).


Conflicts of Interest

None declared.


?

Multimedia Appendix 1

Attitude Social influence-self-Efficacy model (ASE) [69,70] adapted for recovery and Return to Normal Activities (RNA) and Return to Work (RTW) after gynaecological surgery.

[PDF File (Adobe PDF File), 18KB]


?

Multimedia Appendix 2

Example of change objectives of patients.

[PDF File (Adobe PDF File), 25KB]


?

Multimedia Appendix 3

Theoretical methods and practical strategies for recovery and return to normal and work activities.

[PDF File (Adobe PDF File), 42KB]


?

Multimedia Appendix 4

Evaluation questionaire of the eHealth intervention ikherstel.nl for focus group participants.

[PDF File (Adobe PDF File), 305KB]


?

Multimedia Appendix 5

Evaluation questionaire of the eHealth intervention ikherstel.nl for professionals.

[PDF File (Adobe PDF File), 303KB]


?

Multimedia Appendix 6

Screenshot of http://www.ikherstel.nl.

[PDF File (Adobe PDF File), 81KB]


?

Multimedia Appendix 7

Screenshot of http://www.ikherstel.nl.

[PDF File (Adobe PDF File), 206KB]


?

Multimedia Appendix 8

Screenshot of http://www.ikherstel.nl.

[PDF File (Adobe PDF File), 752KB]


?

Multimedia Appendix 9

Screenshot of http://www.ikherstel.nl.

[PDF File (Adobe PDF File), 193KB]


?

Multimedia Appendix 10

Screenshot of http://www.ikherstel.nl.

[PDF File (Adobe PDF File), 225KB]


References

  1. ?Clayton M, Verow P. A retrospective study of return to work following surgery. Occup Med (Lond) 2007 Oct;57(7):525-531 [FREE Full text] [CrossRef] [Medline]
  2. ?Johansen P, Al-Khafagi SK, Th?stesen LM, Lauszus FF, Rasmussen KL. [Analysis of need for sick leave after hysterectomy]. Ugeskr Laeger 2008 Apr 21;170(17):1465-1468. [Medline]
  3. ?Kikuchi I, Takeuchi H, Shimanuki H, Kitade M, Kumakiri J, Kuroda K, et al. Questionnaire analysis of recovery of activities of daily living after laparoscopic surgery. J Minim Invasive Gynecol 2008 Jan;15(1):16-19. [CrossRef] [Medline]
  4. ?Br?lmann HA, Vonk Noordegraaf A, Bruinvels DJ, de Vet RH, Dirksz AA, Huirne JA. Can prolonged sick leave after gynecologic surgery be predicted? An observational study in The Netherlands. Surg Endosc 2009 Oct;23(10):2237-2241. [CrossRef] [Medline]
  5. ?Department of Health. Choosing Health: Making Healthy Choices Easier. London: National Health Service (NHS); 2004.
  6. ?van Oostrom SH, Driessen MT, de Vet HC, Franche RL, Schonstein E, Loisel P, et al. Workplace interventions for preventing work disability. Cochrane Database Syst Rev 2009(2):CD006955. [CrossRef] [Medline]
  7. ?Commissie Psychische Arbeidsongeschiktheid. Aanpak verzuim om psychische redenen. Leidraad 2001 Nov 01.
  8. ?Waddell G, Burton AK. Is Work Good for Your Health and Well-being?. London: The Stationery Office (TSO); 2006.
  9. ?Henderson M, Glozier N, Holland Elliott K. Long term sickness absence. BMJ 2005 Apr 9;330(7495):802-803. [CrossRef] [Medline]
  10. ?Bisgaard T, Klarskov B, Rosenberg J, Kehlet H. Factors determining convalescence after uncomplicated laparoscopic cholecystectomy. Arch Surg 2001 Aug;136(8):917-921. [Medline]
  11. ?Warren L, Ladapo JA, Borah BJ, Gunnarsson CL. Open abdominal versus laparoscopic and vaginal hysterectomy: analysis of a large United States payer measuring quality and cost of care. J Minim Invasive Gynecol 2009 Sep;16(5):581-588. [CrossRef] [Medline]
  12. ?Thiel JA, Kamencic H. Assessment of costs associated with outpatient total laparoscopic hysterectom. J Obstet Gynaecol Can 2006 Sep;28(9):794-798. [Medline]
  13. ?Everett TR, Crawford RA. Laparoscopic assisted vaginal hysterectomy and bilateral salpingo-oophorectomy as a day surgery procedure: a promising option. J Obstet Gynaecol 2010;30(7):697-700. [CrossRef] [Medline]
  14. ?Dirksz AA, Bruinvels DJ, de Vet H, Brolmann HA. Werkhervatting na gynaecologische ingrepen. Scriptie 2008.
  15. ?L?fgren A, Hagberg J, Alexanderson K. What physicians want to learn about sickness certification: analyses of questionnaire data from 4019 physicians. BMC Public Health 2010;10:61 [FREE Full text] [CrossRef] [Medline]
  16. ?Weevers HJ, van der Beek AJ, Anema JR, van der Wal G, van Mechelen W. Work-related disease in general practice: a systematic review. Fam Pract 2005 Apr;22(2):197-204 [FREE Full text] [CrossRef] [Medline]
  17. ?Dasinger LK, Krause N, Thompson PJ, Brand RJ, Rudolph L. Doctor proactive communication, return-to-work recommendation, and duration of disability after a workers' compensation low back injury. J Occup Environ Med 2001 Jun;43(6):515-525. [Medline]
  18. ?Shulman AG, Amid PK, Lichtenstein IL. Returning to work after herniorrhaphy. BMJ 1994 Jul 23;309(6949):216-217. [Medline]
  19. ?Lewin SA, Skea ZC, Entwistle V, Zwarenstein M, Dick J. Interventions for providers to promote a patient-centred approach in clinical consultations. Cochrane Database Syst Rev 2001(4):CD003267. [CrossRef] [Medline]
  20. ?Cheraghi-Sohi S, Bower P. Can the feedback of patient assessments, brief training, or their combination, improve the interpersonal skills of primary care physicians? A systematic review. BMC Health Serv Res 2008;8:179 [FREE Full text] [CrossRef] [Medline]
  21. ?Kinnersley P, Edwards A, Hood K, Ryan R, Prout H, Cadbury N, et al. Interventions before consultations to help patients address their information needs by encouraging question asking: systematic review. BMJ 2008;337:a485 [FREE Full text] [Medline]
  22. ?Haywood K, Marshall S, Fitzpatrick R. Patient participation in the consultation process: a structured review of intervention strategies. Patient Educ Couns 2006 Oct;63(1-2):12-23. [CrossRef] [Medline]
  23. ?Bruegel RB. Patient empowerment--a trend that matters. J AHIMA 1998 Sep;69(8):30-3; quiz 35. [Medline]
  24. ?Aujoulat I, d'Hoore W, Deccache A. Patient empowerment in theory and practice: polysemy or cacophony? Patient Educ Couns 2007 Apr;66(1):13-20. [CrossRef] [Medline]
  25. ?Samoocha D, Bruinvels DJ, Elbers NA, Anema JR, van der Beek AJ. Effectiveness of web-based interventions on patient empowerment: a systematic review and meta-analysis. J Med Internet Res 2010;12(2):e23 [FREE Full text] [CrossRef] [Medline]
  26. ?Cross M. How the internet is changing health care. BMJ 2008;337:a883. [Medline]
  27. ?Randeree E. Exploring technology impacts of Healthcare 2.0 initiatives. Telemed J E Health 2009 Apr;15(3):255-260. [CrossRef] [Medline]
  28. ?de Jong T, Heinrich J, Blatter BM, Anema JR, van der Beek AJ. The feasibility of a web-based counselling program for occupational physicians and employees on sick leave due to back or neck pain. BMC Med Inform Decis Mak 2009;9:46 [FREE Full text] [CrossRef] [Medline]
  29. ?Forkner-Dunn J. Internet-based patient self-care: the next generation of health care delivery. J Med Internet Res 2003 Apr;5(2):e8 [FREE Full text] [CrossRef] [Medline]
  30. ?Hartmann CW, Sciamanna CN, Blanch DC, Mui S, Lawless H, Manocchia M, et al. A website to improve asthma care by suggesting patient questions for physicians: qualitative analysis of user experiences. J Med Internet Res 2007;9(1):e3 [FREE Full text] [CrossRef] [Medline]
  31. ?Gerber BS, Eiser AR. The patient physician relationship in the Internet age: future prospects and the research agenda. J Med Internet Res 2001 Apr;3(2):E15 [FREE Full text] [CrossRef] [Medline]
  32. ?Brug J, Campbell M, van Assema P. The application and impact of computer-generated personalized nutrition education: a review of the literature. Patient Educ Couns 1999 Feb;36(2):145-156. [Medline]
  33. ?Kroeze W, Werkman A, Brug J. A systematic review of randomized trials on the effectiveness of computer-tailored education on physical activity and dietary behaviors. Ann Behav Med 2006 Jun;31(3):205-223. [CrossRef] [Medline]
  34. ?Lustria ML, Cortese J, Noar SM, Glueckauf RL. Computer-tailored health interventions delivered over the Web: review and analysis of key components. Patient Educ Couns 2009 Feb;74(2):156-173. [CrossRef] [Medline]
  35. ?Krebs P, Prochaska JO, Rossi JS. A meta-analysis of computer-tailored interventions for health behavior change. Prev Med 2010 Sep;51(3-4):214-221. [CrossRef] [Medline]
  36. ?Neville LM, Milat AJ, O'Hara B. Computer-tailored weight reduction interventions targeting adults: a narrative systematic review. Health Promot J Austr 2009 Apr;20(1):48-57. [Medline]
  37. ?Miniwatts Marketing Group. Internet World Stats. 2010. Usage and Population Statistics ? URL: http://www.internetworldstats.com/stats.htm [accessed 2012-05-29] [WebCite Cache]
  38. ?Griffiths F, Lindenmeyer A, Powell J, Lowe P, Thorogood M. Why are health care interventions delivered over the internet? A systematic review of the published literature. J Med Internet Res 2006;8(2):e10 [FREE Full text] [CrossRef] [Medline]
  39. ?Alpay LL, Henkemans OB, Otten W, R?vekamp TA, Dumay AC. E-health applications and services for patient empowerment: directions for best practices in The Netherlands. Telemed J E Health 2010 Sep;16(7):787-791. [CrossRef] [Medline]
  40. ?Vitacca M, Mazz? M, Scalvini S. Socio-technical and organizational challenges to wider e-Health implementation. Chron Respir Dis 2009;6(2):91-97. [CrossRef] [Medline]
  41. ?Bartholomew LK, Parcel GS, Kok G. Intervention mapping: a process for developing theory- and evidence-based health education programs. Health Educ Behav 1998 Oct;25(5):545-563. [Medline]
  42. ?Bartholomew LK, Parcel GS, Kok G, Gottlieb NH. Planning Health Promotion Programs: An Intervention Mapping Approach. San Francisco, CA: Jossey-Bass; 2006.
  43. ?Kok G, van Essen GA, Wicker S, Llupi? A, Mena G, Correia R, et al. Planning for influenza vaccination in health care workers: an Intervention Mapping approach. Vaccine 2011 Nov 3;29(47):8512-8519. [CrossRef] [Medline]
  44. ?Samoocha D, Bruinvels DJ, Anema JR, Steenbeek R, van der Beek AJ. Empowerment of disability benefit claimants through an interactive website: design of a randomized controlled trial. BMC Med Inform Decis Mak 2009;9:23 [FREE Full text] [CrossRef] [Medline]
  45. ?Albada A, van Dulmen S, Otten R, Bensing JM, Ausems MG. Development of E-info gene(ca): a website providing computer-tailored information and question prompt prior to breast cancer genetic counseling. J Genet Couns 2009 Aug;18(4):326-338. [CrossRef] [Medline]
  46. ?Prins RG, van Empelen P, Beenackers MA, Brug J, Oenema A. Systematic Development of the YouRAction program, a computer-tailored physical activity promotion intervention for Dutch adolescents, targeting personal motivations and environmental opportunities. BMC Public Health 2010;10:474 [FREE Full text] [CrossRef] [Medline]
  47. ?van Oostrom SH, Anema JR, Terluin B, Venema A, de Vet HC, van Mechelen W. Development of a workplace intervention for sick-listed employees with stress-related mental disorders: Intervention Mapping as a useful tool. BMC Health Serv Res 2007;7:127 [FREE Full text] [CrossRef] [Medline]
  48. ?Vermeulen SJ, Anema JR, Schellart AJ, van Mechelen W, van der Beek AJ. Intervention mapping for development of a participatory return-to-work intervention for temporary agency workers and unemployed workers sick-listed due to musculoskeletal disorders. BMC Public Health 2009;9:216 [FREE Full text] [CrossRef] [Medline]
  49. ?Kristensen TS. Intervention studies in occupational epidemiology. Occup Environ Med 2005 Mar;62(3):205-210 [FREE Full text] [CrossRef] [Medline]
  50. ?Green LW, Kreuter MW. Health Program Planning: An Educational and Ecological Approach. 4th edition. New York: McGraw Hill Professional; 2005.
  51. ?Stichting Informatie Gezondheidszorg. Landelijke Medische Registratie LMR - Verrichtingen. Prismant 2009.
  52. ?Centraal Bureau voor de Statistiek. Women with paid work. Statline databank 2009.
  53. ?Morgan D. Focus Groups as Qualitative Research. Thousand Oaks, CA: Sage; 1997.
  54. ?Barbour RS, Kitzinger J. Developing Focus Group Research: Politics Theory and Practice. London: Sage; 1999.
  55. ?Scientific Software Development. ATLAS.ti version 6.2. Berlin: Scientific Software Development; 2010.
  56. ?van Achterberg T, Huisman-de Waal GG, Ketelaar NA, Oostendorp RA, Jacobs JE, Wollersheim HC. How to promote healthy behaviours in patients? An overview of evidence for behaviour change techniques. Health Promot Int 2011 Jun;26(2):148-162 [FREE Full text] [CrossRef] [Medline]
  57. ?Ritterband LM, Thorndike FP, Cox DJ, Kovatchev BP, Gonder-Frederick LA. A behavior change model for internet interventions. Ann Behav Med 2009 Aug;38(1):18-27. [CrossRef] [Medline]
  58. ?M?ller C, Ottesen M, Kehlet H, Ottesen BS. [Convalescence recommendations after hysterectomy. A study of opinions among Danish physicians]. Ugeskr Laeger 2001 Dec 10;163(50):7043-7047. [Medline]
  59. ?Clayton M, Verow P. Advice given to patients about return to work and driving following surgery. Occup Med (Lond) 2007 Oct;57(7):488-491 [FREE Full text] [CrossRef] [Medline]
  60. ?Shaw L, Segal R, Polatajko H, Harburn K. Understanding return to work behaviours: promoting the importance of individual perceptions in the study of return to work. Disabil Rehabil 2002 Mar 10;24(4):185-195. [Medline]
  61. ?Rosenberger PH, Jokl P, Ickovics J. Psychosocial factors and surgical outcomes: an evidence-based literature review. J Am Acad Orthop Surg 2006 Jul;14(7):397-405. [Medline]
  62. ?Celestin J, Edwards RR, Jamison RN. Pretreatment psychosocial variables as predictors of outcomes following lumbar surgery and spinal cord stimulation: a systematic review and literature synthesis. Pain Med 2009 May;10(4):639-653. [CrossRef] [Medline]
  63. ?Shehmar M, Gupta JK. The influence of psychological factors on recovery from hysterectomy. J R Soc Med 2010 Feb;103(2):56-59. [CrossRef] [Medline]
  64. ?Brouwer S, Krol B, Reneman MF, B?ltmann U, Franche RL, van der Klink JJ, et al. Behavioral determinants as predictors of return to work after long-term sickness absence: an application of the theory of planned behavior. J Occup Rehabil 2009 Jun;19(2):166-174. [CrossRef] [Medline]
  65. ?Oyeflaten I, Hysing M, Eriksen HR. Prognostic factors associated with return to work following multidisciplinary vocational rehabilitation. J Rehabil Med 2008 Jul;40(7):548-554 [FREE Full text] [CrossRef] [Medline]
  66. ?Svajger A, Winding K. Perceptions of possibilities of returning to work with chronic musculoskeletal disorders. Work 2009;32(4):443-454. [CrossRef] [Medline]
  67. ?Lambeek LC, van Mechelen W, Knol DL, Loisel P, Anema JR. Randomised controlled trial of integrated care to reduce disability from chronic low back pain in working and private life. BMJ 2010;340:c1035 [FREE Full text] [Medline]
  68. ?Spelten ER, Sprangers MA, Verbeek JH. Factors reported to influence the return to work of cancer survivors: a literature review. Psychooncology 2002 Mar;11(2):124-131. [Medline]
  69. ?de Vries H, Dijkstra M, Kuhlman P. Self-efficacy: The third factor besides attitude and subjective norm as a predictor of behavioural intentions. Health Educ Res 1988;3:273

    Source: http://www.jmir.org/2012/5/e124/

    emmys emmys torrey smith torrey smith sofia vergara oakland raiders Jessica Lange

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.