



































Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Community
Ask the community for help and clear up your study doubts
Discover the best universities in your country according to Docsity users
Free resources
Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors
The role of mathematical and statistical skills in performance analysis in sports, focusing on defining performance indicators, establishing data reliability, and modelling performances. It also explores various statistical processes used in notational analysis and their effectiveness in enhancing performance prediction.
Typology: Summaries
1 / 43
This page cannot be seen from the preview
Don't miss anything!
Notational analysis – a mathematical perspective.
Mike Hughes, CPA, UWIC, Cyncoed, Cardiff, CF26 2XD.
Abstract
The role of feedback is central in the performance improvement process, and by inference, so is the need for accuracy and precision of such feedback. The provision of this accurate and precise feedback can only be facilitated if performance and practice is subjected to a vigorous process of analysis. Recent research has reformed our ideas on reliability, performance indicators and performance profiling in notational analysis – also statistical processes have come under close scrutiny, and have generally been found wanting. These are areas that will continue to develop to the good of the discipline and the confidence of the sports scientist, coach and athlete. If we consider the role of a performance analyst in its general sense in relation the to the data that the analyst is collecting, processing and analysing, then there a number of mathematical skills that will be required to facilitate the steps in the processes:- i) defining performance indicators, ii) establishing the reliability of the data collected, iii) ensuring that enough data have been collected to define stable performance profiles, iv) determining which are important, v) comparing sets of data, vi) modelling performances and vii) prediction. The mathematical and statistical techniques commonly used and required for these processes will be discussed and evaluated in this paper.
1 Introduction
Recent research has reformed our ideas on reliability, performance indicators and performance profiling in notational analysis – also statistical processes have come under close scrutiny, and have generally been found wanting. These are areas that will continue to develop to the good of the discipline and the confidence of the sports scientist, coach and athlete. If we consider the role of a notational analyst (Fig. 1) in its general sense in relation to the data that the analyst is collecting, processing and analysing, then there a number of mathematical skills that will be required to facilitate the steps in the processes:-
The recent advances made into the research and application of the mathematical and statistical techniques commonly used and required for these processes will be discussed and evaluated in this paper.
2 Mathematical and statistical processes in notational analysis
2.1 Performance indicators
A performance indicator is a selection, or combination, of action variables that aims to define some or all aspects of a performance. Analysts and coaches use performance indicators to assess the performance of an individual, a team, or elements of a team. They are sometimes used in a comparative way, with opponents, other athletes or peer groups of athletes or teams, but often they are used in isolation as a measure of the performance of a team or individual alone. Through an analysis of game structures and the performance indicators used in recent research in performance analysis, Hughes and Bartlett (2002) defined basic rules in the application of performance indicators to any sports. In every case, success is relative, either to your opposition, or to previous performances of the team or individual. To enable a full and objective interpretation of the data from the analysis of a performance, it is necessary to compare the collected data to aggregated data of a peer group of teams, or individuals, which compete at an appropriate standard. In addition any analysis of the distribution of actions across the playing surface must be normalised with respect to the total distribution of actions across the area. Performance indicators, expressed as non-dimensional ratios, have the advantage of being independent of any units that are used; furthermore, they are implicitly independent of any one variable. They also allow, as in the example of bowling in cricket (see Hughes and Bartlett, 2002, p. ), an insight into differences between performers that can be obscure in the raw data. The use of non-dimensional analysis is common in fluid dynamics, which offers empirical clues to the solution of multivariate problems that cannot be solved mathematically. Sport is even more complex, the result of interacting human behaviours; to apply simplistic analyses of raw sports data can be highly misleading. Current research (Hughes et al, 2001; James et al., 2004) is examining how normative profiles are established - how much data are required to reliably define a profile and how this varies with the different types and natures of the data involved in any analysis profile. This area is discussed in detail later. Many of the most important aspects of team performance cannot be ‘teased out’ by biomechanists or match analysts working alone – a combined research approach is needed (Fig. 2.). This is particularly important for information processing – both in movement control and decision making. We should move rapidly to incorporate into such analyses qualitative biomechanical indicators that contribute to a successful movement. These should be identified interactively by biomechanists, notational analysts and coaches, sport-by-sport and movement-by-movement, and validated against detailed biomechanical measurements in controlled conditions. Biomechanists and notational analysts, along with experts in other sports science disciplines –
Coach Athletes
Motor Control
Notational Analyst
Coach Athletes
Biomechanist
Performance Data
Coach Athletes
Performance Analysis Team
Fig. 2. A digital systems approach to the data sharing that the interactive commercial systems have enabled for performance analysts working with coaches and athletes (From Hughes, 2004).
2.2 Analysis of the relative importance of Performance Indicators
Defining performance indicators (PI’s) is based on prior knowledge and research of the effect of relevant factors on relevant dependent variables. However, in many areas, defining performance indicators remains difficult to do accurately. This is due to the complexity of the behaviour of interest as well as the large number of relevant factors. Many factors are often difficult to define and measurements used may be of limited reliability and validity. To be able to comment appropriately (accurately and relevantly) on a given sport, it is necessary to know exactly which performance indicators are relevant to the results. Coaches and, to some extent, notational analysts have traditionally relied upon experience and their own ideas about their sport. This has led to a huge spectrum of performance indicators being used in some sports, particularly soccer. But as databases of multiple sporting events, such as the rugby or the soccer world cups or repeated grand slam events in tennis, are compiled by comprehensive and sophisticated computerised analyses, then comparative evaluation
of PI’s should be possible – using the outcomes of the tournaments as a correlating factor. Two recent pieces of research have given a starting point in the methodology to assess the relative importance of PI’s.
Table 1. Categorisation of the application of different performance indicators in games
Match Classification Biomechanical Technical/Tactical
Always compare with opponents’ data and, where possible, with aggregated data from peer performances.
Compare with previous performances and with team members, opponents and those of a similar standard. Consider normalising data when a maximum or overall value both exists and is important or when inter- individual or intra- individual across-time comparisons are to be made.
Always normalise the action variables with the total frequency of that action variable or, in some instances, the total frequency of all actions.
Luhtanen et al. (2001) analysed offensive and defensive variables of field players and goalkeepers in the EURO 2000 soccer tournament and related the results to the final team ranking in the tournament. All matches (n=31) of the EURO 1996 and 2000 were recorded using video and analysed by three trained observers with a computerised match analysis system. The written definitions of each event (pass, receiving, run with the ball, shot, scoring trial, defending against scoring trial, interception and tackle) were applied in analysing the matches. The quantitative (number of executions) and qualitative (percentage of successful executions) game performance variables were as follows: passes, receivings, runs with ball, scoring trials, interceptions, tackles, goals and goalkeeper’s savings. The total playing times were recorded and the game performance results were standardised for 90 minutes playing time. Team ranking in each variable (quantitative and qualitative) was used as a new variable. The final ranking orders in the EURO 1996 and EURO 2000 tournaments were explained by calculating the rank correlation coefficients between team ranking in the tournament and ranking in the following variables: ranking of ball possession in distance, passes, receiving, runs with the ball, shots, interceptions, tackles and tackles. Selected quantitative and qualitative sum variables were calculated using ranking order of all obtained variables, only defensive variables and only offensive variables. The means and standard deviations of the game performance variables were calculated. Ranking order in each variable was constructed. Spearman’s correlation coefficients (95 % CI) were calculated between all ranking variables describing game performance. In this way the relative importance of the variables were assessed. Although this research was orientated towards explaining the success of the top teams, and did not explore different
apart were deemed to be not significantly different. In sport margins of 2% can mean the difference between winning and coming last (women’s 400m final – Sydney, 2000). It would seem that a simple percentage calculation gives a simple indicator of the absolute differences of the data sets, and these are easily related back to the aims of the research. It was demonstrated that these tests can also lead to errors and confusion if the researchers involved were not specific enough with the definitions of how they defined their percentage differences. Hughes et al (2001) suggested that the following conditions should be applied.
9 The data should initially retain its sequentiality and be cross-checked item against item. 9 Any data processing should be carefully examined as these processes mask original observation errors. 9 The reliability test should be examined to the same depth of analysis as the subsequent data processing, rather than being performed on just some of the summary data. 9 Careful definition of the variables involved in the percentage calculation is necessary to avoid confusion in the mind of the reader, and also to prevent any compromise of the reliability study. 9 It is recommended that the calculation of % differences is based upon
(Σ(mod[V 1 -V 2 ])/ Vmean)*100 %,
(where V 1 and V 2 are variables, Vmean their mean, mod is short for modulus and Σ means ‘sum of’).
This is then used to calculate percentage error for each variable involved in the observation system, and these can be plotted against each variable, and each operator. All the variables should be carefully defined. This will give a powerful and immediate visual image of the reliability tests.
2.4 Establishing the stability of performance profiles
2.41 Empirical methods
It is an implicit assumption in notational analysis that in presenting a performance profile of a team or an individual that a ‘normative profile‘ has been achieved. Inherently this implies that all the variables that are to be analysed and compared have all stabilised. Most researchers assume that this will have happened if they analyse enough performances. But how many is enough? In the literature there are large differences in sample sizes. Hughes Evans and Wells (2001) trawled through some of the analyses in soccer shows the differences (Table 2). They suggested that there must be some way of assessing how data within a study is stabilising. Many research papers in match analysis present data taken from an arbitrarily selected number of performances. Research by Hughes, Wells and Matthews (2000) attempted to ‘validate’ their performance profiles. To establish that a normative profile had been reached, the profiles of 8 matches were compared with those of 9 and 10 matches, using dependent t-tests, for each of the standards of players – no non-parametric software was
available. The results showed that the elite players and the county players did establish a playing pattern that could be said to be reproduced reasonably consistently.
Table 2. Some examples of sample sizes for profiling in sport.
Research Sport N (matches for profile) Reep & Benjamin (1969) Soccer 3, Eniseler et al., (2000) Soccer 4 Larsen et al., (2000) Soccer 4 Hughes et al., (1988) Soccer 8 (16 teams) Tyryaky et al., (2000) Soccer 4 and 3 (2 groups) Hughes (1986) Squash 12, 9 & 6 – 3 groups Hughes & Knight (1993) Squash 400 rallies Hughes & Williams (1987) Rugby Union 5 Smyth et al., (2001) Rugby Union 5 and 5 Blomqvist et al., (1998) Badminton 5 O'Donoghue (2001) Badminton 16, 17, 17, 16, 15 Hughes & Clarke (1995) Tennis 400 rallies O'Donoghue & Ingram (2001)
Tennis 1328<rallies<4300 ( groups)
The recreational players failed to show any real significance at a suitable level, therefore a normal playing pattern was not established. It must be understood that each squash match, even at the same level can differ from one another. A question that arose from this is, how much can they differ before they are suspected that the difference is caused by something other than chance? This was felt to be especially true for the recreational playing standards, where their differences were not caused by chance but by the fact that they have no fixed pattern of play. As players work their way up the different standards of play a set pattern was emerging. The elite players are at a level where a fixed pattern can be established. This method clearly demonstrated that those studies assuming that 5, 6 or 8 matches of performances were enough for a normative profile, without resorting to this sort of test, are clearly subject to possible flaws. It is vital that, in notational analysis, the same care is adopted in examining our validity and reliability techniques. The nature of the data themselves will also effect how many performances are required – 5 matches may be enough to analyse passing in field hockey, would you need 10 to analyse crossing or perhaps 30 for shooting? The way in which the data is analysed also will effect the stabilisation of performance means – data that are analysed across a multi-cell representation of the playing area will require far more performances to stabilise than data that are analysed on overall performance descriptors ( e.g. shots per match). It is misleading to test the latter and then go on to analyse the data in further detail. Evans (1999) aimed to develop a notation system to record the rally outcome variables of badminton, and test system validity and reliability. This system was used to tackle two basic problems that affect the analyst working in this area of performance analysis. The one problem that was examined which is specific to this study was the affect of new match data on the accumulative mean of each variable of the rally ending badminton profile, in an attempt to quantify the matches necessary to establish templates. For Evans (1999) the number of matches required to establish a
is dependent upon the nature of the data and, in particular, the nature of the performers.
Me a n n u m b e r o f r a l l i e s b y g a m e
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Numbe r of ga me s Cumula t ive Me a n ' +10% ' - 10% ' +5% ' - 5% ' +1% ' - 1%
Fig. 3. Example of the % difference plots.
9 The number of matches required to establish a profile of elite women’s movement was dependent on the nature of the data. For elite women, profiles were achieved within three to nine matches (under 10% error) depending on the variables being analysed.
9 The main problem associated with any primary study aiming at establishing previously unrecorded ‘normal’ profiles remains reliability and accuracy. Any future studies that proclaim data as a performance profile should provide supportive evidence that the variable means are stabilising. A percentage error plot showing the mean variation as each match/player is analysed is one such technique. This can be adapted to different sports when analysing profiles/templates of performance.
For the working performance analyst the results provide an estimate of the minimum number of matches to profile an opponent’s rally-end play. Whilst the results may be limited, the methodology of using graphical plots of cumulative means in attempting to establish templates of performance has been served.
2.42 Confidence intervals
The procedure outlined above is expensive in terms of time when collecting data for the first time and is limited in its applicability in many cases due to fluctuations in factors such as team changes, maturation and the fact that some performances never stabilise. James et al. (2004) suggested an alternative approach whereby the specific estimates of population means are calculated from the sample data through confidence limits (CL’s). CL’s represent upper and lower values between which the true (population) mean is likely to fall based on the observed values collected. Calculated CL’s naturally change as more data is collected, typically resulting in the confidence interval (upper CL minus lower CL) decreasing. Confidence intervals (CI’s) were therefore suggested to be more appropriate as performance guides compared to using mean values. Using a fixed value appears to be too constrained due to potential confounding variables that typically affect performance, making prescriptive targets untenable.
Table 3. Mean profiles and 95% confidence limits for the positional clusters of prop, hooker and lock.
Prop ( n = 3)*** Hooker ( n = 1) Lock ( n = 4)** Mean +CL -CL Mean +CL -CL Mean +CL -CL
Successful Tackles 4.01 4.96 3.06 4.25 5.59 2.91 5.73 6.81 4.
Unsuccessful Tackles 0.73 1.12 0.33 0.51 1.01 0 0.76 1.15 0.
Successful Carries 4.25 5.32 3.18 3.51 5.37 1.66 2.20 2.81 1.
Unsuccessful Carries 0.23 0.44 0.01 0.40 0.80 0 0.03 0.08 0
Successful Passes 1.76 2.46 1.06 0.87 1.68 0.06 1.23 1.76 0.
Unsuccessful Passes 0.47 0.98 0 0 0 0 0.20 0.33 0.
Handling Errors 0.33 0.56 0.09 0.27 0.69 0 0.68 0.94 0.
Normal Penalties 0.68 0.97 0.39 0.40 0.79 0 1.12 1.47 0.
Yellow Cards 0.05 0.11 0 0.08 0.24 0 0.09 0.20 0
Tries Scored 0.02 0.07 0 0.10 0.31 0 0 0 0
Successful Throw-ins (^) - - - 9.76 11.85 7.66 - - -
Unsuccessful Throw-
ins - - - 2.94 4.09 1.79 - - -
Successful Lineout
Takes - - - - - - 4.05 4.83 3.
Unsucc. Lineout Takes - - - - - - 0.43 0.71 0.
Successful Restart
Takes - - - - - - 0.65 0.91 0.
Unsucc. Restart Takes - - - - - - 0.59 0.90 0.
From a theoretical perspective, James et al. argued that the use of CI’s can also add significance to the judgement of the predictive potential of a data set, i.e. whether
correctly (Vincent, 1989), a Xi^2 analysis should be applied to data of this nature. As a means of comparison, these analyses are presented in Table 5.
Table 5. Correlation and Xi^2 analysis applied to the intra-operator data from Table 8.
P value 0.000 0.000 0.000 0.000 0. Xi^2 0.069 0.005 0.383 0.165 0. P value 1.000 1.000 0.996 0.999 0.
It is disturbing that, given the % error calculations on the different variables above, that these tests indicate little sensitivity to the differences in the sets of data. It seems puzzling that these tests are so insensitive to differences as large as 5%. At what size of differences do they indicate a significant difference? To examine further the sensitivity of these tests to differences in data sets, the reliability test data were manipulated with increments of 10% changes in values. When these differences were calculated all in the same direction, then the correlation coefficient, and the equivalent for Xi^2 analysis, is always perfect (r=1). Consequently the increments were alternately added and subtracted from the items in descending order in the column of data. The correlation data dropped below the P>0.05 (rcrit= 0.811) level when differences of 40% were added and subtracted to the original data, that is data sets over 30% different would be indicated to be the same (P<0.05) using correlation. The Xi^2 analysis data only showed significant differences (P<0.05) when between 20% and 30% incremental differences had been added and subtracted in the data. A number of the research papers examined used Anova, which is not the correct method for non-parametric data. The textbooks (Vincent, 1995) explain that this is a threat to the validity of the testing process but by how much? There is little in the literature to help with this difficulty. Because of this a comparison of the Kruskal- Wallis analysis and Anova is presented in Table 6.
Table 6. Kruskal-Wallis and Anova applied to the different variables for 5 different sets of data.
Kruskal-Wallis Anova Variables H-value P-value F-value P-value Tackle 2.89 0.576 0.79 0. Pass 6.65^ 0.156^ 4.68^ 0. Kick 4.34 0.361 1.23 0. Ruck 6.03 0.197 1.45 0. Lineout 1.50 0.827 0.25 0.
The surprising fact in the analysis is that neither test indicates that there are any significant differences at the P<0.05 level. Both tests indicate that the biggest differences are in the variable the pass, while the tackle and the ruck had the largest differences (9.4% and 10.3% respectively) when calculated by % measurements. This is because these tests measure the overall variance about the median and mean – and may therefore miss one relatively large difference measurement if all the others are close to the mean or median. It is interesting to manipulate the data, as before, to
examine what levels of errors will cause these tests to indicate that significant differences are present (Table 7). The F value is a ratio of between-group variances to those within the groups. Consequently, although the means of the data are moving further apart, because the variation within each group varies as much, more in some cases, the F values do not approach any closer to the critical values. Consequently using these tests does not appear to be practical for comparison studies. Another common problem in notation studies is with the amounts of data collected for the reliability study – have enough repetitions been completed? How many are required? Perhaps the small amounts of data here are contributory towards the poor performance of these tests.
Table 7. Manipulation of some sample data to test the sensitivity of Kruskal-Wallis and Anova for comparing multiple sets of non-parametric data.
Kruskal-Wallis Anova % changes H-value P-value F-value P-value 5% 2.89 0.576 2.84 0. 10% 6.65 0.156 0.86 0. 15% 4.34 0.361 1.57 0. 20% 6.03 0.197 0.97 0.
Although Atkinson and Neville (1998) have produced a definitive summary of reliability techniques in sports medicine, the only attempt that has been attempted to make recommendations in the use of techniques in performance analysis for solving some of the common problems associated with these types of data was that by Nevill et al (2002). This is written very much from a statistician’s point of view particularly extolling the virtues of the high power software GLIM. They concluded that investigating categorical differences in discrete data using traditional parametric tests of significance (e.g. ANOVA, based on the continuous symmetric normal distribution) is inappropriate. More appropriate statistical methods were promoted based on two key discrete probability distributions, the Poisson and binomial distributions. In the opinion of Nevill et al. (2002) the first approach is based on the classic x^2 test of significance (both the goodness-of-fit test and the test of independence). The second approach adopts a more contemporary method based on log-linear and logit models using the statistical software GLIM. Although investigating relatively simple one-way and two-way comparisons in categorical data, both approaches result in very similar conclusions, which is comforting to the ‘old guard’ notational analysts who have been using x^2 for the last few decades. However, as soon as more complex models and higher-order comparisons are required, the approach based on log-linear models is shown to be more effective. Indeed, when investigating factors and categorical differences associated with binomial or binary response variables, such as the proportion of winners when attempting decisive shots in squash or the proportion of goals scored from all shots in association football, logit models become the only realistic methods available. Nevill et al. concluded that with the help of such log-linear and logit models, greater insight into the underlying differences or mechanisms in sport performance can be achieved.
2.6 Modelling performances
2.61 Empirical models
Hughes (1985) established a considerable database on different standards of squash players. He examined and compared the differences in patterns of play between recreational players, country players and nationally ranked players, using the computerised notational analysis system he had developed. The method involved the digitization of all the shots and court positions, and these were entered via the QWERTY keyboard. Hughes (1986) was able then to define models of tactical patterns of play, and inherently technical ability, at different playing levels in squash. Although racket developments have affected the lengths of the rallies, and there have been a number of rule changes in the scoring systems in squash, these tactical models still apply to the game today. This study was replicated, with a far more thorough methodology, for the women’s game by Hughes et al. (2001). Fuller (1988) developed and designed a Netball Analysis System and focused on game modelling from a data base of 28 matches in the 1987 World Netball Championships. There were three main components to the research - to develop a notation and analysis system, to record performance, and to investigate the prescience of performance patterns that would distinguish winners from losers. The system could record how each tactical entry started; the player involved and the court area through which the ball travelled; the reason for each ending; and an optional comment. The software produced the data according to shooting analysis; centre pass analysis; loss of possession; player profiles; and circle feeding. Fuller's (1988) intention of modelling play was to determine the routes that winning, drawing and losing teams took and to identify significantly different patterns. From the results Fuller was able to differentiate between the performances of winning and losing teams. The differences were both technical and tactical. The research was an attempt to model winning performance in elite netball and more research needed in terms of the qualitative aspects i.e. how are more shooting opportunities created. the model should be used to monitor performance over a series of matches not on one-off performances. Treadwell, Lyons, Potter (1991) suggested that match analysis in rugby union and other field games has centred on game modelling and that their research was concerned with using the data to predict game content's of rugby union matches. They found that clear physiological rhythms and strategic patterns emerged. They also found that at elite level it was possible to identify key "windows" i.e. vital "moments of chronological expectancy where strategic expediency needs to be imposed." It appears that international matches and successful teams generate distinctive rhythms of play which can exhibit a team fingerprint or heartbeat”. Lyons (1988) had previously analysed 10 years of Home Nations Championship matches to build up a database, and from this was able to predict actions for a match to limits of within 3 passes and 2 kicks. Empirical models enable both the academic and consultant sports scientist to make conclusions about patterns of play of sample populations of athletes within their sport. This, in turn, gives the academic the potential to examine the development and structures of different sports with respect to different levels of play, rule changes,
introduction of professionalism, etc. The consultant analyst can utilise these models to compare performances of peer athletes or teams with whom the analyst is working.
2.62 Dynamic systems
Modelling human behaviour is implicitly a very complex mathematical exercise, which is multi-dimensional, and these dimensions will depend upon 2 or 3 spatial dimensions together with time. But the outcomes of successful analyses offer huge rewards, as Kelso (1999) pointed out:-
If we study a system only in the linear range of its operation where change is smooth, it’s difficult if not impossible to determine which variables are essential and which are not.
Most scientists know about nonlinearity and usually try to avoid it.
Here we exploit qualitative change, a nonlinear instability, to identify collective variables, the implication being that because these variables change abruptly, it is likely that they are also the key variables when the system operates in the linear range.
Recent research exploring mathematical models of human behaviour led to populist theories categorised by ‘Catastrophe Theory’ and ‘Chaos Theory’.
2.621 Catastrophe Theory and Chaos Theory Catastrophe Theory Kirkcaldy (1983) described catastrophe theory as:
"... a descriptive model... which allows us to better appreciate the manner in which multi-dimensional systems operate and to make predictions of the behaviour of the systems under scrutiny."
The mathematical model was originated by Thom (1975) and later modified by Zeeman (1975; 1976). Kirkcaldy (1983) used the model to provide a possible explanation of how explosive effects can accompany small changes in arousal to produce an optimum level of performance or a sudden decrement in performance. It is concerned with the methods of attaining equilibrium states in qualitative mathematical language. Thom's (1975) 3-dimensional model of catastrophe attempted to explain the relationship between cognitive and somatic anxiety and athletic performance, and predict performance from this. Catastrophe theory predicts a negative linear relationship between the cognitive anxiety and the performance, but that the somatic anxiety also plays a role. Hardy and Fazey (1987) hypothesised that if somatic anxiety increases towards optimum while cognitive anxiety is low then performance will be facilitated.
Chaos Theory The ability to predict performance is inherent in the process of effective planning., but is very difficult to do accurately. Errors in statistical methods of predicting are often attributed to forecasting error but chaos theory suggests that those errors imply that
performance. Developing the skills and imagination to tease these links out of these fascinating areas will make very rewarding research in the future?
2.622 Critical Incident Technique At the First World Congress of Notational Analysis of Sport (1991), Downey talked of rhythms in badminton rackets, athletes in “co-operation” playing rhythmic rallies, until there was a dislocation of the rhythm (a good shot or conversely a poor shot) – a ‘perturbation’ – sometimes resulting in a rally end situation ( a ‘critical incident’), sometimes not. A good defensive recovery can result in the re-establishment of the rhythm. This was the first time that most of us had considered different sports as an example of a multidimensional, periodic dynamic system. The term ´critical incident´ was first coined by Flanagan (1954) in a study designed to identify why student pilots were exhausted at flight school.
"The critical incident technique outlines procedures for collecting observed incidents having special significance and meeting systematically defined criteria."
The critical incident technique is a powerful research tool but as with other forms of notating behaviour there are limitations inherent in the technique. Flanagan (1954) admitted that "Critical incidents represent only raw data and do not automatically provide solutions to problems" and Burns (1956) was even more critical:
"The indiscriminate use of the critical incident technique in the establishment of success criteria... can only result in fetish collection of data which describes everything and explains nothing."
Another limitation of the technique is the total dependence on the reporters' opinions and this subjective element inherent in the use of the technique is often stated as a disadvantage. But Flanagan also pointed to the advantages of such a technique:
"The critical incident technique, rather than collecting opinions, hunches and estimates, obtains a record of specific behaviours from those in the best position to make the necessary observations and evaluations."
These opinions sound very much like the debates that have surrounded notational analysis within the halls of sports science over the last decade. Research in sport has addressed some of these issues and notation and movement analysis systems have been developed that can overcome some of these disadvantages. Since Downey’s suggestions in 1991, some researchers have investigated the possibilities that analysing ‘perturbations’ and ‘critical incidents’ offer. McGarry and Franks (1997) applied further research to tennis and squash. They derived that every sporting situation contains unique rhythmical patterns of play. This behavioural characteristic is said to be the stable state or dynamic equilibrium. The research suggested that there are moments of play where the cycle is broken and a change in flow occurs. Such a moment of play is called a ‘perturbation’. It occurs when either a poorly executed skill or a touch of excellence forces a disturbance in the stability of the game. For example, in a game of rugby this could be a bad pass or immediate change of pace. From this situation the game could unfold one of two ways; either the flow of play could be re- established through defensive excellence or an attacking error, or it could result in loss of possession or a try that would end the flow. When the perturbation results in a
loss of possession or a try, it was then defined as a ‘Critical Incident’, sometimes the perturbation may be ‘smoothed out’, by good defensive play or poor attacking play, and not lead to a critical incident. A more in-depth approach is essential in order to derive a system for analyzing the existence of perturbations. In order to do so, entire phases of play must be analysed and notated accordingly. Applying McGarry and Franks’ (1995) work on perturbations in squash, Hughes et al. (1997) attempted to confirm and define the existence of perturbations in soccer. Using twenty English league matches, the study found that perturbations could be consistently classified and identified, but also that it was possible to generate specific profiles of variables that identify winning and losing traits. After further analyses of the 1996 European Championship matches (N=31), Hughes et al. (1997) attempted to create a profile for nations that had played more than five matches. Although supporting English League traits for successful and unsuccessful teams, there was insufficient data for the development of a comprehensive normative profile. Consequently, although failing to accurately predict performance it introduced the method of using perturbations to construct a prediction model. By identifying 12 common attacking and defending perturbations that exist in English football leading to scoring opportunities, Hughes et al. (1997) had obtained variables that could underpin many studies involving perturbations. These twelve causes were shown to occur consistently, covering all possible eventualities and had a high reliability. Although Hughes et al. (1997) had classified perturbations; the method prevented the generation of a stable and accurate performance profile. In match play, teams may alter tactics and style according to the game state; for instance a team falling behind may revert to a certain style of play to create goal-scoring chances and therefore skew any data away from an overall profile. In some instances, a perturbation may not result in a shot, owing to high defensive skill or a lack of attacking skill. Developing earlier work on British league football, Hughes et al. (2001) analysed how the international teams stabilise or ‘smooth out’ the disruption. Analysing the European Championships in 1996, attempts were made to identify perturbations that did not lead to a shot on goal. Hughes et al. (2001) refined the classifications to 3 types of causes; actions by the player in possession, actions by the receiver and interceptions. Inaccuracy of pass accounted for 62% of the player in possession variables and interception by the defence accounted for the vast majority of defensive actions (68%). Actions of the receiver (12%) were dominated by a loss of control; however these possessions have great importance because of the increased proximity to the shot (critical incident). Conclusions therefore focussed on improvements in technical skill of players, however with patterns varying from team to team, combining data provides little benefit for coaches and highlights the need for analysing an individual teams ‘signature’. Squash is potentially an ideal sport for analyzing perturbations, and as such has received considerable attention from researchers. It is of a very intense nature and is confined to a small space. The rhythms of the game are easy to see, and the rallies are of a length (mean number of shots at elite level = 14; Hughes and Robertson , 1997) that enables these rhythms and their disruption i.e. perturbations and critical incidents. In defining perturbations, it may help to understand the reasons for this occurrence, consider the simplest model of a dynamic oscillating system. What are the main parameters and do the equivalent variables in sports contests conform to the same behaviour?