Analyzing challenges associated with the adoption of longitudinal studies in Work and Organizational Psychology

 

Analisando desafios associados à adoção de pesquisas longitudinais em Psicologia Organizacional e do Trabalho

 

Analizando desafíos asociados a la adopción de investigaciones longitudinales en Psicología Organizacional y del Trabajo

 

 

Gardênia da Silva Abbad^I; Mary Sandra Carlotto^II

^IUniversidade de Brasília, Brasília, DF, Brasil
^IIUniversidade do Vale do Rio dos Sinos, São Leopoldo, RS, Brasil

Endereço para correspondência

 

 

---

ABSTRACT

This article aims to analyze the main theoretical and methodological challenges associated with the use of longitudinal research designs in Work and Organizational Psychology. Longitudinal designs with three or more points of measurement are rare in the literature. Critical points to be observed include: definition of the interval between first and last measurement of variables, time needed for the effect of an intervention to become observable, quantity of measurements over time, time period between measurements, management of participant drop outs, and analysis of alternative explanations for the effects of interventions or programs. The technical recommendations offered include the use of designs with multiple measurements of independent and dependent variables, such as external variables that threaten the internal and external validity of study conclusions.

Keywords: research method; longitudinal design; Work and Organizational Psychology.

---

RESUMO

Este artigo objetiva analisar os principais desafios teóricos e metodológicos associados ao uso de delineamentos longitudinais de pesquisa em Psicologia Organizacional e do Trabalho. Delineamentos longitudinais com três ondas de mensuração ou mais são raros na literatura da área. Entre os pontos críticos a serem observados está a definição do intervalo de tempo entre a primeira e última mensuração das variáveis, o tempo necessário para que o efeito de uma intervenção se torne observável, a quantidade de mensuração ao longo do tempo, o tempo entre mensurações, o manejo da perda seletiva de sujeitos e a análise de explicações alternativas aos efeitos de intervenções ou programas. Entre as recomendações técnicas estão o uso de delineamentos com múltiplas mensurações, tanto das variáveis independentes e consequentes, como de variáveis externas que ameaçam a validade interna e externa das conclusões desses estudos.

Palavras-chave: método de pesquisa; delineamento longitudinal; Psicologia Organizacional e do Trabalho.

---

RESUMEN

Ese artículo objetiva analizar los principales desafíos teóricos y metodológicos asociados al uso de diseños longitudinales de investigación en Psicología Organizacional y del Trabajo. Diseños longitudinales con tres o más olas de mensuración son raros en la literatura del área. Entre los puntos cruciales que se observarán, está la definición del intervalo de tiempo entre la primera y la última mensuración de las variables, el tiempo necesario para que el efecto de una intervención sea observable, la cantidad de mensuraciones a lo largo del tiempo, el tiempo entre mensuraciones, el manejo de la pérdida selectiva de sujetos y el análisis de explicaciones alternativas para los efectos de intervenciones y programas. Entre las recomendaciones técnicas, está la utilización de diseños con múltiples mensuraciones, tanto de las variables independientes y consecuentes como de las variables externas que amenazan la validad interna y externa de las conclusiones de estos estudios.

Palabras-clave: método de investigación; diseño longitudinal; Psicología Organizacional y del Trabajo.

---

 

 

This article aims to analyze the main theoretical and methodological challenges associated with the use of the concept of time in researches with longitudinal design, as well as analyze advantages, limitations and possibilities of advances in production of knowledge in Work and Organizational Psychology (WOP) about constructs of interest, based on theoretical and methodological approaches sensitive to the dynamic and mutable nature of processes, constructs and associations between constructs over time.

This article has as specific objectives: (a) analyze the concept of time, identifying its uses and meanings in everyday language and in technical language adopted by researchers in organizational studies; (b) analyze the main challenges associated with the construction of theories and research hypotheses that take into account the dynamic nature of constructs and relationships of interest; (c) analyze and present examples and methodological challenges in longitudinal researches referring to the definition of minimum amount of repetitions of measurements of variables, to the choice of strategies to deal with loss of subjects and to monitoring or control of external threats to the validity of research conclusions; as well as (d) point the main contributions from the adoption of mixed studies with multiple information sources, time series and field quasi-experiment in longitudinal studies.

Many are the methodological challenges faced by those who study human behavior concerning organizations and work, especially in situations in which studied variables, due to their nature, require, for their proper description, comprehension and/or explanation, two measurements (or observations) or more over a certain time period. Time plays an important role in organizational reality. Organizations compete to operate in the market, pressuring teams and workers with increasingly tight schedules and deadlines (Sonnentag, 2012). People are currently living and working with the concept of simultaneous time that incorporates other senses, in addition to linear time, which include increased number of activities within one same unit of time (Tonelli, 2008).

Although time has acquired a central role in work activities and organizations, its meaning is not dully contemplated in the construction of empirical studies and theories (Sonnentag, 2012). Cross-sectional studies, defined as research designs that adopt one or more samples of population and collect information at a single moment are predominant in WOP (Shaufnnessy, Zechmeister, & Zechmeister, 2012). Longitudinal research designs are those in which one same sample is observed (studied) at more than one moment.

Ployart and Vanderberg (2010) state that the vast majority of theories tested in applied organizational and social sciences still adopts cross-sectional research designs, through which inferences of associations between two variables or more are carried out as if they were static, not changing with time. Organizational theories are explicitly or implicitly longitudinal, because the processes they refer to comprehend relationships between variables belonging to multiple levels of analysis and that vary over time. However, researchers seem to ignore the dynamic nature of phenomena and relationships of interest when planning and conducting researches in WOP.

Several aspects of organizational behavior vary over time, with some varying less quickly (culture, values) and others suffering frequent variations (productive performance, absenteeism, stress, people's opinions on situations and events). These are variations of interest for researchers who want to study behavior at micro (individual), meso (groups, teams, work process) and macro (organization and society) levels.

In WOP, many concepts adopted by researchers express individual or collective dispositions, which cannot be dully comprehended or explained through a single observation, as they do not refer to unique occurrences, but to complex relations between behaviors, conditions and reasons, which change with time and situations. The dispositional nature of many concepts, such as that of attitudes or performance, points to a need for longitudinal studies and long-term follow-up, since transitions and changes that these variables go through over time are vital for the comprehension of important phenomena of interest. Cross-sectional studies with measurement of variables at one single moment are not sensitive to transitions and changes that occur in human behavior and in its work contexts.

According to Taris and Kompier (2014), in the last three decades longitudinal researches have become increasingly frequent in both occupational health psychology and medicine. Many researchers consider longitudinal designs to be more robust and adequate to WOP studies, as they enable more robust conclusions about relations between preceding (or predicting) and consequent variables than cross-sectional ones. Field longitudinal experimental or quasi-experimental studies (with multiple measurements of independent (IV) and dependent variables (DV) before, during and after an intervention) are robust as well as more indicated for studies that intend to investigate cause and effect relations, manipulating independent variables, observing the effects of these manipulations on DV and controlling alternative explanations to effects that threaten the validity of causality inferences.

Turnover, work performance, commitment with the organization, motivation, learning, professional development, emotions and affections at work, wellbeing and health, occupational diseases, for instance, are phenomena that require, in an ideal sense, longitudinal researches, as they vary in time, are subject to the influence of innumerous variables of the context, located at different levels of analysis. The description, comprehension, prediction and explanation of this type of phenomenon require theories and research designs that enable the study of these variations and, at least, the formulation of hypotheses on the nature, magnitude and direction of relations between variables throughout time. Cross-sectional research designs are not sensitive to these changes because they are based on a single observation of variables. Longitudinal designs, on the other hand, are more sensitive and more adequate to the comprehension of changes over time. What are, after all, longitudinal studies? What are the fundamental differences between cross-sectional and longitudinal studies?

Longitudinal research is that which aims to capture the dynamic nature of processes and relations between variables, and comprehends a minimum of three observations of at least one substantive construct of interest (Kelloway & Francis, 2013; Ployart & Vanderberg, 2010; Taris & Kompier 2014). Some authors, however, also call as longitudinal those studies that collect data on only two occasions in time (T1 and T2) (Menard, 2002; Trochim, 2006). When all constructs of interest are measured at all moments of the research, the study is called panel study. The adoption of longitudinal research designs is based on the assumption that the variability associated with one or more constructs at a given moment in time can be very different from the variability associated with these very same constructs at other moments. Longitudinal studies allow for the construction and adoption of theories and hypotheses on possible changes in relations between constructs, which can only be detected through repeated observations in different moments and situations.

Cross-sectional researches, on the other hand, are conducted from static theories, which generally requires only one observation of variables of interest, which, excepting in experiments, hinders the formulation of valid inferences of causality or prediction relations between variables (preceding and consequent), due to lack of control of alternative explanations to these effects. Cross-sectional studies bring little comprehension of how a variable changes with time, because the static nature of unique observation and measurement does not allow the formulation of valid and robust conclusions on changes in variables and relations between variables of interest. Longitudinal research is, therefore, a research approach in which the researcher collects data repeatedly over time. Studies with only one pre and post-test (T1 and T2) would not be, according to this definition, longitudinal researches (Ployart & Vanderberg, 2010; Van Ness, Fried, & Gill, 2011).

Given the importance of time in WOP researches and of challenges faced by researchers while elaborating and testing theories that take into account the dynamic nature of variables and relations between variables of interest, and that they are sensitive to typical transitions and changes of constructs from the WOP area, this article analyzes meanings and uses of the word time in everyday language and in technical language of organizational studies, based on techniques from the philosophy of language that allow analyzing the logic of use of concepts in everyday language and of conceptual reviews oftentimes proposed by technical language, which cause conceptual, theoretical and methodological confusions. The analysis of these uses will facilitate the identification of critical points to be considered by researches when they elaborate theories and chose methods for research in WOP.

Techniques adopted by researchers (Harzem & Miles, 1978) and philosophers such as Ryle (1949) and Peters (1958) include the polar principle (what is and what is not expressed by a certain concept), which enables the identification of conceptual confusions that result in inadequate choices of research methods. This approach, in this article, will serve as a support to the analysis of adequacy of longitudinal studies to the nature of concepts underlying studied phenomena and variables.

This article will present, later on, an analysis of uses of the time construct in organizational researches, challenges associated with the construction of theories and longitudinal research methods in WOP; use of mixed, multiphase and multilevel methods in longitudinal studies; and, in the course of the text, longitudinal studies that exemplify the adoption of longitudinal theories and research designs, indicating critical points associated with choice of designs compatible with the nature and complexity of constructs and relations between variables, pointing advantages and limitations of this type of research in WOP.

 

Time: Meanings and uses

This section analyzes meanings attributed to the concept of time in everyday language and in technical language of organizational studies, aiming to identify similarities and differences between uses, as well as identify confusions or reviews of the concept.

In American English (Cambridge Dictionary), time is used as (a) measure of existence (seconds, minutes, hours, days, months, years, etc); (b) specific period (a certain interval of seconds, minutes, hours, days, weeks, months, years, etc., during which something is happening or is necessary or is available); (c) clock measurement (a specific moment of the day, as expressed in hours, minutes and shown in the clock; or specific moment of the day, week, month or year); (d) occasion: a moment of the day, week, month, year that is adequate to a specific activity or refers to something that is expected to happen; and (e) historical period: a period in history or prior to 'its' time.

According to Aurélio dictionary, the word time in Portuguese (tempo) originates from Latin, being derived from tempus and temporis, which mean division of duration in instant, second, minute, hour, day, month, year, etc. The noun tempo possesses diverse meanings, with some of them being similar to those attributed to the word time in English. They are: constant and eternal series of instants; arbitrary measurement of the duration of things; a certain age; deadline; delay; season; age (relatively to certain circumstances in life, to state of things, to habits, to opinions); a certain age when a fact happened or a character existed; occasion, opportunity; precise instant of the movement in which one of its parts should be performed.

In the technical language adopted in organizational studies the concept of time has been used to refer to measures and occasions. Among meanings linked to time as scale or measure, there are several methodological questions, including: time intervals between related events (time-lags); time elapsed between one measurement and another; age when something happened or historical context in which relations between variables occurred or suffered changes; deadline so a change can be observed; duration of organizational, group, individual processes; and between meanings related to the occasion there is highlight to: exact moment when a variable begins to influence or stop influencing another; instant or moment when one should observe and measure a relation between variables, perceptions and attitudes about time (psychological measure). These meanings have been adopted to discuss methodological and theoretical challenges of organizational studies, as well as to distinguish constructs of uses given by everyday language to the word time.

The psychological meaning of time, however, cannot be confounded with that of physical time. This use of the concept as psychological construct differs from the meanings of the word time in everyday language, in both Portuguese and English. It is a technical use for the concept that refers to people's perceptions, attitudes, feelings, emotions and affections in relation to time. This use is associated with psychological measures that require, in addition to physical time measurement, instruments and units of measure capable of detecting constructs such as feelings of time urgency, emotional and cognitive responses to time pressure, an individual's affections and feelings face challenges as he searches for balance when dividing time for work, family, friends and leisure.

In WOP, there are theoretical and methodological questions that require the use of measurement of time as a physical and psychological construct. Theoretical questions include: (a) prior definition of the way that variables of interest and relations between these variables change as time passes; (b) when (on what occasions) these changes can be observed, (c) how much time is necessary for variations in these processes and relationships to become observable; (d) how much time of observation is necessary for their cycles (increase, stabilization or decrease) to be observed and described; (e) how many times and at what moments these changes should be measured in order to enable the description of the direction and format of relations between variables of interest; and (f) how much time and on what occasions context variables need to be described, all external to studied relationships, which can threaten the validity of causality inferences in experiment or quasi-experiment, or result predictions in surveys. These theoretical questions need to be answered or predicted by the researcher before the planning of a longitudinal research. Otherwise, the researcher will not observe the dynamic character of changes occurred in variables and relationships over time. The method is (or should be) chosen according to the nature of research problems and the context in which the study is developed.

Time is generally only a measure or scale to represent change processes, according to Ployart and Vanderberg (2010), which, nevertheless, should be considered by the researcher during the elaboration of theories and choice of research methods. The analysis of the concept of time shows that the word time, as a physical measure, cannot be mistaken for the concept of 'cause' of change processes studied in WOP. Time is not a cause, is a measure. Important constructs in the area, such as turnover, burnout, stress, commitment, performance and learning at work, do not change or develop because of time, but over time. Next, uses of the concept in organizational researches will be discussed.

 

Uses of the concept of time in organizational researches

In organizational studies the construct time has been used to: (a) investigate time-related psychological constructs, (b) study time-sensitive organizational processes, (c) specify time intervals adequate to detection and measurement of a process or result and (d) analyze processes in different contexts or focuses (historical, economic and other) that allow for an investigation of processes and events that have occurred in the past or at the present moment; and at different levels of analysis, which enable an investigation of processes that occur or have occurred inside the organization, in the broader organizational environment, and in less comprehensive contexts such as teams and individuals (Sonnentag, 2012).

Time-related constructs include: time urgency, time pressure, time-family conflict or polychronicity, temporal focus (Sonnentag, 2012) and the team's temporal leadership (degree to which team leaders define deadlines, synchronize the team's behaviors and allocated temporal resources). In these cases, time is a psychological construct of central interest in this type of research. In this case, the time construct is not used as a measure or occasion on which something is measured or observed.

Examples of time-sensitive processes that can be mentioned are occupational stress, learning at work, performance of individuals and teams, turnover, affective commitment and burnout. There are physical, social and psychological stressors associated with tasks and roles played by the individual, as well as time pressure, which oftentimes becomes chronic and tend to produce negative cognitive and affective states that vary with time, which, for this reason, are not detectable by cross-sectional studies (Ford et al., 2014). Due to this relationship subject to changes of direction and intensity over time is that stress has been increasingly studied through three-wave longitudinal designs (T1, T2 and T3); all measurements are applied at these three moments (full panel), aiming at the construction of theories and conduction of researches compatible with the temporal nature of relationships between these variables (Ford et al., 2014).

In a three-wave longitudinal study with 262 volunteer sports administrators, Cuskelly and Boag (2001) examined the influence of time on the relationship between organizational commitment and perceptions about the operation of sports committees to predict the turnover of members of sports organizations in Australia. Said study is an example of three-wave longitudinal research - T1 (beginning of sports season), T2 (end of season) and T3 (beginning of next year's season) - and full panel because the questionnaires referring to turnover, organizational commitment and perception about the operation of sports committees were applied at those three moments.

Cuskelly and Boag (2001) is also an example of study in which researchers had to define the time interval of the research (a year), and did so according to sports competitions seasons in Australia, respecting the characteristics of the context in which relations between constructs were studied, and, besides, specified the moment when the questionnaires were applied, making them coincide with three critical points of the season (beginning, end, and beginning of the subsequent season) and comprehending a full cycle of competitions. The theory and method adopted in that research proved compatible with the nature of studied constructs, showing that methodological decisions on time should be preceded by hypotheses and results of other researches that provide empirical support to hypotheses and assumptions on changes expected in studied processes.

Westman and Eden (1997) used repeated observations with the aim of investigating the effect of vacation on stress and Burnout. Considering that an ideal synchronism has not been identified in the literature, stress and Burnout were assessed on four occasions: 6 weeks before vacation, 3 days before vacation, at the beginning of the second week of vacation, and 3 days and 3 weeks after vacation. Results indicated a decline in Burnout levels during vacation and a return to levels prior to vacation, assessed by the time of the second measurement after vacation. Comparing the two measurements prior to vacation, no significant difference was found. However, return to work, assessed in the third week after vacation, showed return to Burnout levels similar to that of the six weeks before vacation. The study reveals the phenomenon's transience to the extent that, with the removal of stressors, Burnout levels decrease. In this case, repeated measurement enabled the production of relevant knowledge about the subject. If the researchers had adopted a cross-sectional design they would not have captured these variations in burnout levels.

In studies about causal relationships between variables there are big theoretical and methodological challenges, including: defining how much time is necessary for a variable to produce effects on another and, in the case of reciprocal relationships, how much time is necessary for this change of sense to occur and what time intervals between observations are necessary to detect these changes. Many organizational phenomena characterize changes in cause and effect relationships over time. These variations can occur in the sense of relations between variables (what was cause becomes effect and vice-versa) and in the magnitude of these relationships.

An example of study that revealed reciprocal relationships between variables was the one conducted by Conway and Coyle-Shapirto (2012). Said research aimed to investigate the reciprocal relationship between perceptions about compliance with psychological contract and the employee's performance (sales made and goal achievement), and analyze moderation effects between variables in a sample of 146 salespeople. The longitudinal design adopted in this research used four measurement waves (T1, T2, T3 and T4). The time interval between T1 and T4 was 16 months. Results evidenced: (a) reciprocal connections between performance and compliance with the psychological contract with the organization, (b) increase in the magnitude of the relationship between performance and compliance with the psychological contract over time and (c) moderation effect of the quality of social exchange relationship variable (POS) on reciprocal connections between sales made and sales goal achievement and perceptions about compliance with the psychological contract at closer moments, but not at more distant points in time. This type of research shows the importance of longitudinal design as a strategy that facilitates the improvement of theories on constructs related to phenomena sensitive to the passage of time. The research evidences advantages of this design over cross-sectional studies, which are not sensitive to variations in relationships between variables over time.

When selecting research design, choosing or constructing theory, it is necessary, according to Sonnentag (2012), to define in advance how much time is needed for an X variable to influence another (Y); how much times is needed for this effect to become observable and for Y to remain stable; when Y begins to change again and when Y begins to disappear gradually. In organizational studies, answers to these questions have not been properly approached yet in theorizations or research methods.

To Sonnentag (2012), in addition to three ways of focusing time in researches - (a) psychological construct, (b) time-sensitive organizational processes and (c) time intervals adequate to detection and measurement of a process or result -, there is a fourth use of the concept of time, which refers to the analysis of organizational processes in different contexts or focuses (historical, economic and others) and at different levels of analysis.

That fourth way of focusing time in organizational studies is about defining it as temporal context or environmental stimulus. It refers to the way that the historical, economic, political and cultural context outside of the organization affects phenomena of interest and, besides, that internal processes (past and current) occurred inside the organization or in the broader organizational environment and in less comprehensive contexts, linked to teams and individuals, affect studied relationships. Absenteeism, turnover and career are examples of processes under the influence of economic and political variables of the macro level (environment), of variables belonging to the meso (groups, units and work processes) and micro levels (psychologic), which need to be considered by the researcher when constructing, choosing theories and research designs.

 

Theoretical and methodological challenges of longitudinal researches

This section analyzes some critical points associated with the adequate use of longitudinal research designs in WOP. Challenges in making these designs more sensitive to changes with the passage of time include: (a) specification of the nature of relationships between independent variables (preceding), dependent variables (consequent) and mediating variables, defining for each relationship whether it is static (can be measured at one single point in time) or dynamic (needs to be measured at three moments at least); (b) elaboration of a theory that points precisely what variables are expected to change, the causes of these changes, the dynamic nature of the relationship of variables of interest over time, indicating how, why and in what circumstances variables of interest change with time; (c) development of a theory that indicates the format taken by the relationship between variables over time (linear -positive or negative) or nonlinear (plateau effect, sudden break, multiple increases and decreases); (d) definition of levels of analysis of changes (intra-group and between units of analysis) and of predictive variables of these intra and between-level changes; (e) definition of the minimum amount of repetitions of variable measurement, in order to capture changes in processes and variables in the course of a certain time interval; (f) choice of strategies to deal with loss of subjects from a measurement moment to subsequent ones (Ployart & Vanderberg, 2010) and (g) monitoring or control of alternative explanations or external threats to the validity of research conclusions.

The first four points concern the importance of constructing theories and research hypotheses that take into account the dynamic nature of constructs and relationships of interest, as well as matters linked to the time necessary for certain effects to become observable, grow, stabilize, so the research design can capture, describe or predict variations. Decisions on methodological strategies to be adopted in longitudinal studies are not generalizable from a construct or theme to another, because changes in relationships between variables depend on the studied phenomenon and context.

In the reviewed literature, there is no consensual answer to facing the first methodological challenge (definition of the number of measurements necessary for the characterization of the study as a longitudinal one). Studies with only two waves of observation or measurement are not regarded as efficient longitudinal studies, because they are little sensitive to changes in relationships between variables over time. This is the opinion of authors mentioned earlier (Ployart & Vanderberg, 2010; Taris & Kompier, 2015). However, three observations of at least one variable of interest seems to be a useful recommendation, since designs with only two measurement waves (T1 and T2) are insufficient and little sensitive to changes occurred in relationships between variables of interest over time for many reasons, including: (a) the fact that, in studies with two waves, every single change from Time 1 to Time 2 is linear (straight line), which allows detecting the presence of other formats (nonlinear) of the change over time, and, additionally, (b) the fact that two observations do not enable a precise analysis of alternative explanations to obtained results, which increases threats to the validity of conclusions on changes occurred in variables and relationships of interest over a certain time interval.

Another essential question in the adoption of longitudinal design is the decision on the interval and amount of repetitions of measurements of variables of interest. These decisions, rarely, are determined by pragmatic reasons (Taris & Kompier, 2014). Time should correspond to the theoretical basis about the occurrence of the phenomenon. In case it is much shorter than the actual interval necessary to detect the influence of the causal variable over the effect, chances are that the explanatory variable has not had yet enough time to affect the effect variable. On the other hand, if the interval between the occurrence of the cause and the effect is very long, the effect of exposure to the explanatory variable may have disappeared. When time intervals are too short or too long, chances of detecting the effect of the explanatory variable over the effect can decrease. Thus, the magnitude of longitudinal effects may vary strongly according to the duration of the interval used to analyze the relationship between causative variable and effect.

Dormann and Zapf (2002) compared results of one year, every two years, and four years, and found that two years at least were necessary for the effects of work stressors over mental health to appear. The lightest effect happened within one year. De Lange, Taris, Kompier, Houtman and Bongers (2004), using one, two and three-year periods, reported that longitudinal effects between work stressors and health (depression, emotional exhaustion and job satisfaction) were stronger for a one-year interval. By the end of 2013, these two studies had been cited 500 times but its value to subsidize the choice of a certain time interval is limited. Neither decisions on ideal time interval between observations waves, nor study conclusions should be generalized to other types of studied stressors and outcomes. This example highlights the importance of developing in advance specific theory on changes expected in constructs and relationships between variables before planning, choosing methods and conducting the research.

Frese and Zapf (1994) carried out a study using five types of development trajectories for processes usually studied in Occupational Health Psychology. For instance, workers can quickly adapt to some stressors, indicating that exposure effects can have short duration and that a short interval between study waves is necessary to detect such effects. Other work factors that may affect health in the long term are referred to by Ford et al. (2014).

It is important to highlight that there are no general rules on time interval between measurements. The option for a certain time interval between waves must necessarily consider the type of causes and consequences that are being investigated and the context of the process that is being assessed (Taris & Kompier, 2014). According to the authors, there should be a clear justification as to the choice of time interval between waves, and the latter should be based on the theoretical model about the impact between variables.

In the absence of solid foundations to the choice of a certain time interval between waves (measurement moments) of a study, researchers can opt for including multiple waves with relatively short time intervals between measurements (Taris & Kompier, 2014). When time is unknown or cannot be reasonably assumed, researchers must preferably employ multiphase projects in which measures are taken from the same group of participants at many points in time (Ployart & Vanderberg, 2010).

In addition to those points, there are questions about levels of analysis at which changes in relationships between variables occur in a certain time interval: individuals, teams, organizations (Ployart & Vanderberg, 2010). For a longitudinal design to be able to detect these variations, it is necessary to collect repeated measures from variables of interest in the same units of observation (individuals, teams, organizations), as well as to investigate the contribution of variables measured in these units of analysis in the prediction of the effect variable.

According to Ployart and Vanderberg (2010), between statistical techniques capable of: (a) describing intra-unit variations of analysis in longitudinal studies, there is the analysis of variance for repeated measures (with the assumption that intra-unit variation has the same format); (b) capturing variations between units, there is the RCM - random coefficient modeling); and (c) testing differences between these coefficients, there is the recommendation of RCM and LGM (latent growth modeling. Besides these strategies, it is possible to include in analysis models predicting variables to explain intra and inter-unit changes, adopting multi-level regression models. For greater details on statistical techniques capable of describing and predicting changes in longitudinal, see Ployart and Vanderberg (2010).

As for the sixth aspect to consider in the planning and conduction of longitudinal researches is the handling of loss of subjects from a measurement moment to subsequent ones. Loss of participants in longitudinal studies can threaten a study's internal and external validity and limit its capacity of making inferences on relationships between preceding and consequent variables. In this sense, efforts towards reducing segment loss bias are an important research priority (Robinson, Dennison, Wayman, Pronovost, & Needham, 2007). Systematic review carried out by the authors about retention strategies identified that studies that combined varied approaches presented smaller losses of participants. Results indicate that most used strategies are: involving the community in the study; creating a sense of identity of participants with the study; explaining in detail study objectives, including potential risks and benefits to participants; providing reminders to participants; promoting benefits or incentives to participants; providing expense reimbursement.

When planning a longitudinal research, determination of an optimal number of cases that take into account loss of subjects is recommended. According to Ployart and Vanderberg (2010), the researcher must be prepared for losing around 50% from the first to the second occasion of measurement. To discover reasons associated with loss of subjects, one can investigate the predictors of said loss through techniques for statistical analysis of missing data such as MCA (missing completely random) and MAR (missing at random).

The seventh question to be considered by the researcher when planning longitudinal studies refers to monitoring or control of threats to the study's internal and external validity, due to the possibility of existence, in the research context, of alternative explanations to changes occurred in variables and relationships between variables of interest. These externalities can affect the internal validity of conclusions on relations of causality (in the case of experiments or quasi-experiments) and prediction between variables (in the case of correlational and prediction researches), since, in the interval between waves of application of measurements or observations in a study, other events can confound or compete with the explanatory or predictive variable in the explanation or prediction of the effect variable (dependent or criterion). These concurring factors threaten the internal validity of study conclusions, since the intensity of the effect exerted by an explanatory variable on the result variable may have been biased by underestimation or overestimation.

Workers are not passive subjects in relation to situational stimuli but can make attempts to change work situations. Between the waves of a study, they can, for instance, try to change the content of their job, their job conditions or working hours (Wrzesniewski & Dutton, 2001). These decisions, not variables manipulated or observed in the study, are what, in situations like that, would explain the variability of the consequent or effect variable. Variations in the organization's external and internal context can also contaminate the results of a research and confound the analysis of relations between study variables. For this reason, if is not possible to control these externalities through field experiments or quasi-experiments, it is necessary to monitor these external variables by means of prior identification of alternative explanations to studied relations, based on analysis of the study context during research planning, and monitoring (measurement) of these variables during the time interval of the study and analysis of the influence of these externalities on variables and relationships of interest.

The identification of external variables that threaten the internal validity of studies can be performed with support from the definition of temporal context or environmental stimulus, proposed by Sonnentag (2012), which preconizes the analysis of the influence exerted by the historical, economic, political and cultural context that is external and internal to the organization of phenomena of interest, at different levels of analysis (e.g., individuals, teams). In this way, longitudinal research includes in its design the study of the context's influence on variables and relationships of interest.

Use of time series design for description, follow-up and assessment of relationships between independent and consequent variables, as well as analysis of the influence of external variables or alternative explanations to results, when adopted, would facilitate the identification of threats to the study's internal and external validity, as well as the analysis of the influence of these factors on variables of interest (Shadish, Cook, & Campbell, 2002). For this monitoring to be possible, it is oftentimes necessary to search for information from secondary sources and archival data. In this sense, the adoption of mixed methods and multiple sources of information seems to be the most adequate strategy towards overcoming challenges posed by the longitudinal analysis of relations between relevant variables.

 

Mixed studies with multiples sources in longitudinal studies

Analysis of data collected at many points in time can involve different sources of information and data, as well as interpretation of qualitative patterns (Saldanã, 2003), analysis techniques for images and visual graphs (Brown, McGuire, Beck, Peterson, & Mooney, 2007), and/or statistical approaches for repeated measures, as well as analysis of variance for repeated measures, such as RCM and LGM (Ployart & Vanderberg, 2010).

Mixed-method research involves collection, analysis and integration of quantitative data and qualitative data inside a single study or multiple phases of a research design (Creswell & Plano-Clark, 2013). This approach, according to Creswell (2010) and Creswell and Plano-Clark (2013), brings advantages from qualitative and quantitative methods, enable the use of several sources and comparison of results obtained by means of different measures based on what individuals say and feel (perceptions, opinions, narratives), on what individuals do (behaviors, performances, actions) and on objective marks left by the individual, which are registered on documents, databases, magazines, records (files, information systems, performance results, epidemiological data, texts, images).

A systematic literature review conducted by Plano-Clark et al. (2014) identified an expressive upward trend for longitudinal mixed studies. The study, conducted in 2012 on PubMed database, analyzed 32 articles representing 30 different areas, including psychology, nursing, public health, health services, gerontology, mental health and education. Most articles performed intervention assessments such as efficacy, impact, and feasibility. In the other ones, the objective was to describe or identify association between variables or the outcome of an experience over time. The justification for using the design was the need to combine methods to assess an intervention better, to corroborate results, to capture the complexity of the phenomenon and to enrichen the interpretation of a type of result with that of the other type.

Mixed designs (quanti and quali), by enabling the combination of two methods, can promote advances in the comprehension of behavior patterns and in the elaboration of explanatory and predictive models of variables of interest in the WOP area. Abbad, Puente-Palacios and Gondim (2014) analyzed some challenges in the adoption of mixed methods and described the research by Rodrigues (2011), which, in addition to serving as an example of successful use of this type of design, indicates some benefits from the approach in WOP studies.

Time series with multiple observations: Field quasi-experiments

According to Shadish, Cook and Campbell (2002), the most robust designs include natural experiments and quasi-experiments conducted outside of laboratories, among which there are time series with multiple tests of variables before, during and after interventions or programs. This type of study allows predicting, identifying, comprehending, preventing and/or minimizing the effect of the experiment's external variables.

Time series studies, also called historical series studies, have as objective to analyze the behavior of a phenomenon over time (Latorre & Cardoso, 2011). It is a combination of data associated with uniform time periods such as, for instance, month, day, week or any fixed window of time to be defined according to the event of interest. Statistically, it is defined as a collection of samples sequentially done during a time period, with a characteristic dependency between neighboring samples, making the analysis of this dependency relevant. Time series analysis is a method to attempt to understand the time series' behavior in order to define and study the structure or cause that has generated the series (Barbieri & Machline, 2006; Morettin & Toloi, 1985). Initially, there is an attempt to model the studied phenomenon and then describe the series' behavior, making estimates and, lastly, identify what factors influence the series' behavior, aiming to define cause and effect relations (Latorre & Cardoso, 2001). Time series studies are still scarce in Occupational Health Psychology, especially when it comes to WOP, in general, but tend to grow due to their importance in the establishment of policies and actions aimed at reducing or stop certain aggravations, and at assessment demands of social programs in terms of final value or social return, whose effects need to be assessed and monitored for a long time. Although time series design has advantages, such as low cost and quick execution (Cunha, Blank, & Boing, 2009), it should be used carefully to monitor confounding variables and follow up effects of interventions (programs) over time.

This type of information (Palmeira, 2000) collected from secondary sources is not always reliable and consistent, and can, for this reason, compromise research validity. For this reason, observation of the following aspects is recommended when choosing this type of complex longitudinal design: (a) adopting primary and secondary sources of information about variables of interest; (b) choosing indicators of analysis and follow-up of effects of programs or interventions that are valid, consistent and reliable; and (c) using qualitative and quantitative mixed method for the analysis of time series so as to increase the design's sensitivity to the detection of changes occurred in relations between variables (direction, sense, relationship format -linear, nonlinear) and possible alternative explanations present in the research context that threaten the longitudinal study's internal and external validity.

A study that adopted strategy of analysis of alternative explanations to relations between variables in a field quasi-experiment, based on secondary data collected in time series, was that conducted by Mourão and Borges-Andrade (2005), in a successful research that assessed a governmental program in Brazil. Another example of time series analysis is the time-trend study of work leaves for mental and behavioral disorders among nurses, conducted by Falavigna and Carlotto (2013), in which data were obtained from secondary sources (records on absenteeism and diseases that led to work leave). Time series studies can be adopted to investigate phenomena and changes of interest and monitor threats to the validity of inferences on causality or prediction relations between variables.

Many phenomena studied by researchers in the WOP require the adoption of longitudinal research designs due to the dynamic nature of relationships between variables of interest. However, theories not always take into account changes that occur in relationships and contexts with time. There is evidence that longitudinal studies enable the enhancement of theories on phenomena of interest, because these designs, more sensitive to variations in magnitude, direction and sense of relationships between variables allow identifying robust evidence of these changes and formulating more valid inferences on these relations.

Nevertheless, for these complex designs to be applicable it is necessary that theories and research hypotheses take into account changes and indicate the time needed for detection of effects, for description of variations in direction, sense and magnitude of relationships between variables, for the format of relationships (linear, nonlinear, abrupt variations, others) and for levels of analysis (individual, group, organization) to be approached in the study.

Critical points to be considered during the planning of a longitudinal research include: (a) specification of the nature of relationships between independent variables (preceding), dependent variables (consequent) and mediating variables, defining for each relationships whether it is static (can be measured at only one point in time) or dynamic (needs to be measured at three moments at least); (b) elaboration of a theory that precisely indicates what variables are expected to change, the causes of these changes, the dynamic nature of the relationship between variables of interest over time, indicating how, why and in what circumstances variables of interest change with time; (c) development of a theory that indicates the format taken by the relationship between variables over time (linear - positive or negative) or nonlinear (plateau effect, sudden break, multiple increases and decreases); (d) definition of levels of analysis of changes (intra-group and between units of analysis) and of predictive variables of these intra and inter-level changes; (e) definition of minimum amount of repetitions of measurements of variables in order to capture changes in processes and variables throughout a certain time interval; (f) choice of strategies to deal with loss of subjects from a measurement moment to subsequent ones (Ployart & Vanderberg, 2010) and (g) monitoring or control of alternative explanations to external threats to the validity of research conclusions, as well as investigation of the influence they exert on variables of interest over time.

The researches chosen to illustrate the use of longitudinal designs in this article were conducted in natural environment which aim at describing relationships between variables and adopt prediction models in which preceding and consequent elements are measured and observed predominantly by means of questionnaires. The present study found no quasi-experiment or field quasi-experiment, neither assessment of interventions or programs, excepting the study by Mourão and Borges-Andrade (2005). The research sample of articles includes scientific articles published in work and organizational psychology national and international magazines within a long period, between 2000 and 2016. This study did not perform a systematic literature review on WOP longitudinal studies because the articles were chosen with the aim illustrating the use of complex longitudinal designs and extracting from them critical points related to the successful planning and execution of this type of study.

The searches led to the finding of several articles that adopted only two measurements (waves) and that, even so, were called longitudinal, which indicates lack of consensus on the minimum number of waves that distinguish these designs from cross-sectional ones. Seemingly, longitudinal studies in WOP are still incipient, probably due to the complexity of theoretical and methodological challenges and high costs (time, staff and financial resources) necessary for the conduction of this type of investigation in natural environment. Because of all advantages and potentialities of longitudinal designs in WOP a greater effort from researchers will be necessary so as to improve theories on time-sensitive phenomena, through multiple waves of measurement of variables of interest; to investigate the time necessary for detection of the variation pattern of intra and inter-units of analysis, as well as identify and monitor alternative explanations present in the context of longitudinal studies.

The adoption of interrupted time series, of multivariate and multilevel analysis models for changes in relationships between variables over time, mixed methods and multiple sources of information seems to be strategies more adequate to increase the internal and external validity of inferences on changes occurred in this relationships with time.

In short, the analysis of theoretical challenges in longitudinal researches suggests that WOP researchers need to enhance their theories and hypotheses by making them more sensitive to the dynamic nature of processes and changes occurred over time in phenomena of interest. For longitudinal designs to be able to capture changes in relationships between variables of interest it is necessary to observe and measure them at three moments at least, because using only two measurements (T1 and T2) allows the identification of solely linear relations between variables of interest, restricting analysis models and the search for more precise explanations on the nature of changes in relationships. The use of mixed methods is important in longitudinal studies because of the influence of externalities (alternative explanations) that threaten the validity of causality or prediction inferences between variables. Combined use of qualitative and quantitative data in longitudinal studies might enrichen analyses and interpretations of results, as well as the monitoring of variables not controlled or manipulated by the study.

Practical application of longitudinal designs in assessment of programs and interventions in organizations and work is recommended, especially when effects of these actions are observable in the medium and long terms. In these cases, the use of field quasi-experimental designs associated with analyses of time series to follow up external variables (out of the program's control) are recommended as they facilitate the identification of contribution of the program's activities for the achievement of expected results and the identification of potential alternative explanations to results of the intervention.

 

References

Abbad, G., Puente-Palacios, K., & Gondim, S. M. G. (2014). Abordagens metodológicas em Psicologia Organizacional e do Trabalho. Revista Brasileira de Psicologia, 1(2),71-78.         [ Links ]

Barbieri, J. C., & Machline, C. (2006). Logística hospitalar: Teoria e prática. São Paulo: Saraiva.         [ Links ]

Brown, C. G., McGuire, D. B., Beck, S. L., Peterson, D. E., & Mooney, K. H. (2007). Visual graphical analysis: A technique to investigate symptom trajectories over time. Nursing Research, 56(3),195-201. doi: 10.1097/01.NNR.0000270029.82736.5a        [ Links ]

Conway, N., & Coyle-Shapiro, J. A. M. (2012). The reciprocal relationship between psychological contract fulfillment and employee performance and the moderating role of perceived organizational support and tenure. Journal of Occupational and Organizational Psychology, 85(2),277-299. doi: 10.1111/j.2044-8325.2011.02033.x        [ Links ]

Creswell, J. W., & Plano-Clark, V. L. (2013). Pesquisa de métodos mistos (2a ed.). Porto Alegre: Penso.         [ Links ]

Creswell, J. W. (2010). Projeto de pesquisa: Métodos qualitativo, quantitativo e misto (3a ed.). Porto Alegre: Artmed.         [ Links ]

Cunha, J. B., Blank, V. L. G., & Boing, A. F. (2009). Tendência temporal de afastamento do trabalho em servidores públicos (1995-2005). Revista Brasileira de Epidemiologia, 12(2),226-236. doi: 10.1590/s1415-790x2009000200012        [ Links ]

Cuskelly G., & Boag A. (2001). Organisational commitment as a predictor of committee member turnover among volunteer sport administrators: Results of a time-lagged Study. Sport Management Review, 4(1),65-86. doi: 10.1016/s1441-3523(01)70070-8        [ Links ]

De Lange, A. H., Taris, T. W., Kompier, M. A. J., Houtman, I. L. D., & Bongers, P. M. (2004). The relationships between job characteristics and mental health: Examining normal, reversed and reciprocal relationships in a 4-wave study. Work & Stress, 18(2),149-166. doi: 10.1080/02678370412331270860        [ Links ]

Dormann, C., & Zapf, D. (2002). Social stressors at work, irritation, and depression: Accounting for unmeasured third variables in a multi-wave study. Journal of Occupational and Organizational Psychology, 75(1),33-58. doi: 10.1348/096317902167630        [ Links ]

Falavigna, A., & Carlotto, M. S. (2013). Tendência temporal de afastamento do trabalho por transtornos mentais e comportamentais em enfermeiros (1998-2008). Revista Psicologia: Organizações e Trabalho, 13(3),363-371.         [ Links ]

Frese, M., & Zapf, D. (1994). Action as the core of work psychology: A German approach. In H. C. Triandis, M. D. Dunnette & J. M. Hough (Eds.), Handbook of industrial and organizational psychology (2nd ed., Vol. 4, pp. 271-340). Palo Alto: Consulting Psychology Press.         [ Links ]

Ford, M. T., Matthews, R. A., Wooldridge, J.D., Mishra, V., Kakar, U. M., & Strahan, S. R. (2014). How do occupational stressor-strain effects vary with time? A review and meta-analysis of the relevance of time lags in longitudinal studies. Work & Stress: An International Journal of Work, Health & Organisations, 28(1),9-30. doi: 10.1080/02678373.2013.877096        [ Links ]

Harzem, P., & Miles, T. R. (1978). Conceptual issues in operant psychology. New York: Wiley & Sons.         [ Links ]

Kelloway, E. K., & Francis, L. (2013). Longitudinal research and data analysis. In L. E. Tetrick, M. Wang & R. R. Sinclair (Eds.), Research methods in occupational health psychology: Measurement, design, and data analysis (pp. 374-393). New York: Routledge.         [ Links ]

Latorre, M. R. D. O., & Cardoso, M. R. A. (2001). Análise de séries temporais em epidemiologia: Uma introdução sobre os aspectos metodológicos. Revista Brasileira de Epidemiologia, 4(3),145-152. doi: 10.1590/s1415-790x2001000300002        [ Links ]

Menard, S. (2002). Longitudinal research. Quantitative applications in the social sciences. Thousand Oaks: Sage.         [ Links ]

Morettin, P. A., & Toloi, C. M. C. (1985). Previsão de séries temporais. São Paulo: Atual.         [ Links ]

Mourão, L., & Borges-Andrade, J. E. (2005). Avaliação de programas públicos de treinamento: Um estudo sobre impacto no trabalho e na geração de empregos. Organizações e Sociedade, 33(12),13-38. doi: 10.1590/s1984-92302005000200001        [ Links ]

Palmeira, G. (2000). Epidemiologia. In S. Rozenfeld (Org.), Fundamentos da vigilância sanitária (pp. 15-48). Rio de Janeiro: Fiocruz.         [ Links ]

Peters, R. S. (1958). The concept of motivation. London: Routledge & Kegan Paul.         [ Links ]

Plano-Clark, V. L., Anderson, N., Wertz, J. A., Zhou, Y., Schumacher, K., & Miaskowski, C. (2014). Conceptualizing longitudinal mixed methods designs: A methodological review of health sciences research. Journal of Mixed Methods Research, 1-23. doi: 10.1177/1558689814543563        [ Links ]

Ployart, R. E., & Vanderberg, R. J. (2010). Longitudinal research: The theory, design, and analysis of change. Journal of Management, 36(1),94-120. doi: 10.1177/0149206309352110        [ Links ]

Robinson, K. A., Dennison, C. R., Wayman, D. M., Pronovost, P. J., & Needham, D. M. (2007). Systematic review identifies number of strategies important for retaining study participants. Journal of Clinical Epidemiology, 60(8),757-765. doi: 10.1016/j.jclinepi.2006.11.023        [ Links ]

Rodrigues, A. C. A. (2011). Trabalhador entrincheirado ou comprometido? Delimitação dos vínculos do indivíduo com a organização. (Tese de doutorado não publicada). Universidade Federal da Bahia, Salvador.         [ Links ]

Ryle, G. (1949). The concept of mind. London: Hutchinson.         [ Links ]

Saldanã, J. (2003). Longitudinal qualitative research: Analyzing change through time. Walnut Creek: AltaMira Press.         [ Links ]

Shadish, W. R., Cook, T. D., & Campbell, D. T (2002). Experimental and quase experimental designs for generalized causal inference. Boston: Houghton Mifflin Company.         [ Links ]

Shaufnnessy, J. J., Zechmeister, E. B., & Zechmeister, J. S. (2012). Metodologia de pesquisa em psicologia (9a ed.). Porto Alegre: AMGH.         [ Links ]

Sonnentag, S. (2012). Time in organizational research: Catching up on a long neglected topic in order to improve theory. Organizational Psychology Review, 2(4),361-368. doi: 10.1177/2041386612442079.         [ Links ]

Taris, T. W., & Kompier, M. (2014). Cause and effect: Optimizing the designs of longitudinal studies in occupational health psychology. Work & Stress: An International Journal of Work, Health & Organisations, 28(1),1-8, doi: 10.1080/02678373.2014.878494        [ Links ]

Taris, T. W., & Kompier, M. (2015). Challenges in longitudinal designs in occupational health psychology. Scandinavian Journal of Work, Environment and Health, 29(1),1-4. doi: 10.5271/sjweh.697        [ Links ]

Tonelli, M. J. (2008). Sentidos do tempo e do tempo de trabalho na vida cotidiana. Organizações & Sociedade, 15(45),207-216. doi: 10.1590/s1984-92302008000200016        [ Links ]

Trochim, W. M. (2006). The research methods knowledge base. Retrieved from http://www.socialresearchmethods.net/kb/index.php        [ Links ]

Van Ness, P. H., Fried, T. R., & Gill, T. M. (2011). Mixed methods for the interpretation of longitudinal gerontologic data: Insights from philosophical hermeneutics. Journal of Mixed Methods Research, 5(4),293-308. doi: 10.1177/1558689811412973        [ Links ]

Westman, M., & Eden, D. (1997). Effects of a respite from work on burnout: Vacation relief and fade-out. Journal of Applied Psychology, 82(4),516-527. doi: 10.1037/0021-9010.82.4.516        [ Links ]

Wrzesniewski, A., & Dutton, J. E. (2001). Crafting a job: Revisioning employees as active crafters of their work. Academy of Management Review, 26(2),S179-201. doi: 5465/amr.2001.4378011        [ Links ]

 

 

Endereço para correspondência:
Gardênia da Silva Abbad
Universidade de Brasília, Instituto de Psicologia, Departamento de Psicologia Social e do Trabalho, Campus Universitario, Instituto Central de Ciências-Sul, Instituto de Psicologia, Programa de Pós-Graduação em Psicologia Social e do Trabalho - PSTO
Asa Norte
Brasília, DF, Brasil 70910-900
Email: gardenia.abbad@gmail.com

Recebido em: 04/07/2016
Primeira decisão editorial em: 12/08/2016
Versão final em: 28/08/2016
Aceito em: 03/09/2016