1 / 22

CITATIONS AS INDICATORS IN TS

CITATIONS AS INDICATORS IN TS. Daniel Gile daniel.gile@yahoo.com www.cirinandgile.com. AIMS OF THIS PRESENTATION. THIS PRESENTATION IS OFFERED IN THE FRAMEWORK OF RESEARCHER TRAINING RESEARCH CONTEXT IS: RESEARCH ON TS AS AN OBJECT OF STUDY

wauna
Download Presentation

CITATIONS AS INDICATORS IN TS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CITATIONS AS INDICATORS IN TS Daniel Gile daniel.gile@yahoo.com www.cirinandgile.com D.Gile citations as indicators

  2. AIMS OF THIS PRESENTATION THIS PRESENTATION IS OFFERED IN THE FRAMEWORK OF RESEARCHER TRAINING RESEARCH CONTEXT IS: RESEARCH ON TS AS AN OBJECT OF STUDY TS IS A COMPLEX ENTITY, HENCE THE USE OF INDICATORS IN ITS STUDY THE CONCEPT OF INDICATORS IS EXPLAINED USES OF CITATIONS AS AN INDICATOR IN THE STUDY OF TS ARE EXPLAINED METHODOLOGICAL ISSUES WHICH ARE RELEVANT TO OTHER EMPIRICAL STUDIES ARE DISCUSSED D.Gile citations as indicators

  3. INDICATORS INDICATORS ARE MEASURABLE UNITS (or at least countable units) WHICH ARE USED TO MEASURE (or at least detect/identify) ASPECTS OF ENTITIES WHICH ARE DIFFICULT TO MEASURE DIRECTLY BECAUSE THEY ARE COMPLEX AND/OR DIFFICULT TO DEFINE OPERATIONALLY D.Gile citations as indicators

  4. EXAMPLES OF INDICATORS - FOR HEALTH: BODY TEMPERATURE, BLOOD PRESSURE… - FOR NATIONAL WEALTH: GNP - FOR LEVEL OF CRIMINAL ACTIVITY: NUMBER OF CRIMINAL TRIALS, NUMBER OF COMPLAINTS TO THE POLICE… - FOR TRANSLATION QUALITY: NUMBER OF ERRORS AND OMISSIONS, NUMBER OF COMPLAINTS FROM USERS - FOR QUALITY OF TRANSLATOR TRAINING PROGRAM: PROPORTION OF GRADUATES WHO FIND WORK AS TRANSLATORS D.Gile citations as indicators

  5. IMPORTANT FEATURES OF INDICATORS (1) - VALIDITY (DO THEY MEASURE WHAT THEY ARE SUPPOSED TO MEASURE?) - CONVENIENCE AND COST OF ACCESS (HOW LONG DOES IT TAKE TO MEASURE THEM? HOW MUCH MONEY/MANPOWER etc. DOES IT COST?) - EASE/COST OF PROCESSING (ONCE YOU HAVE THEM, IN ORDER TO DO SOMETHING USEFUL WITH THEM, HOW MUCH EFFORT AND HOW MANY RESOURCES ARE REQUIRED?) D.Gile citations as indicators

  6. IMPORTANT FEATURES OF INDICATORS (2) - COVERAGE (WHAT PROPORTION OF THE RELEVANT PHENOMENON IN THE ENTITY TO BE STUDIED DO THEY COVER?) - SENSITIVITY (HOW MUCH OF THE “SIGNAL” OR PHENOMENON THEY ARE SUPPOSED TO PICK UP THEY ACTUALLY DO PICK UP?) - SELECTIVITY (DO THEY PICK UP THE TARGET SIGNAL OR PHENOMENON ONLY, OR DO THEY ALSO PICK UP OTHER SIGNALS WHICH INTERFERE WITH THE EXPLORATION PROCESS?) COMPLAINTS FOR CRIMINAL ACTIVITY? VAL/COV/SENS/SEL COMPLAINTS OF TRANSLATION USERS? VAL/COV/SENS/SEL PROPORTION OF GRADUATES WHO FIND EMPLOYMENT? VAL/COV/SENS/SEL D.Gile citations as indicators

  7. CITATIONS WHEN INVESTIGATING TS AS AN OBJECT OF STUDY: WRITING NORMS IN SCIENCE IMPOSE REFERENCES TO AUTHORS AND WORKS “USED” THESE TAKE THE FORM OF CITATIONS Operational definition: EXPLICIT REFERENCE IN A RESEARCH TEXT TO ANOTHER AUTHOR’S PUBLISHED TEXT i.e. NAME OF AUTHOR AND PARTICULAR TEXT WITH REFERENCES D.Gile citations as indicators

  8. CITATIONS AS AN INDICATOR OF INFLUENCE VALID? EASE AND COST OF ACCESS? EASE AND COST OF PROCESSING? COVERAGE? SENSITIVITY? SELECTIVITY? D.Gile citations as indicators

  9. IN THE FIELD CONSIDERED VALID WIDELY USED By GOVERNMENTS, SOCIOLOGISTS OF SCIENCE… IN TS: COVERAGE INSUFFICIENT SENSITIVITY PERHAPS INSUFFICIENT CORRELATION BETWEEN NUMBER OF CITATIONS AND ACTUAL INFLUENCE PERHAPS NOT STRONG ENOUGH D.Gile citations as indicators

  10. TENTATIVE METHODOLOGICAL IMPROVEMENT - 3 LEVELS OF INFLUENCE OF A CITED TEXT? L1: In passing, as illustration, historical reference, NO VISIBLE IMPACT on citing author’s thought or work, judging by the citing text L2: SOME IMPACT: Clear signs in citing text that citing author has thought about the cited text significantly or is building on it L3: CENTRAL in citing text EVALUATE INFLUENCE OF CITED TEXTS IN CITING TEXT D.Gile citations as indicators

  11. 3 LEVELS OF INFLUENCE: IS IT A PROGRESS? PERHAPS TOO MUCH VARIABILITY? PERHAPS DIFFICULT TO CATEGORIZE? HAVE TO CHECK IF NOT GOOD ENOUGH, IMPROVE DEFINITION IF NOT GOOD ENOUGH, GIVE IT UP CHECK IN CLASS WITH PAPERS SEE % OR AGREEMENT, DISAGREEMENT IF OK, COMPARE RESULTS WITH 3 LEVELS AND WITHOUT D.Gile citations as indicators

  12. 3 LEVELS OF INFLUENCE: TESTS? A NUMBER OF TESTS WITH COLLEAGUES TAKE PUBLISHED TEXT ASK PANEL OF RESEARCHERS TO ASSESS LEVEL CHECK CONVERGENCE IF CONVERGENCE, OK, IMPLEMENT (IF COLLECTION COST NOT TOO HIGH) IF NOT ENOUGH CONVERGENCE - TRY TO IMPROVE OPERATIONAL DEFINITION - OR INFER THAT THIS IMPROVEMENT, HOWEVER ATTRACTIVE CONCEPTUALLY, DOES NOT REALLY WORK D.Gile citations as indicators

  13. OPERATIONALIZING CITATION ANALYSISTO MEASURE AUTHOR INFLUENCE - DETERMINE SAMPLING METHOD AND SAMPLE SIZE (TYPE OF CITING TEXTS, PERIOD, LANGUAGES, GEOGRAPHIC AND SOCIOLOGICAL IDENTITY…) IF THESE CONSIDERATIONS ARE NOT TAKEN UP, PROBLEM OF REPRESENTATIVITY • PROCESSING (SIMPLE COUNTS? PROPORTIONS? NUMBER OF TIMES CITED OR NUMBER OF PAPERS IN WHICH CITED?...) D.Gile citations as indicators

  14. WHAT QUESTIONS CAN CITANAL HELP WITH?INITIAL CONTRIBUTIONS - WHO ARE THE MOST INFLUENTIAL AUTHORS AT TIME t? IN TS? IN A PARTICULAR BRANCH OF TS? ON A PARTICULAR TOPIC? WITH A PARTICULAR METHODOLOGY? - SAME QUESTION ABOUT GROUPS, “SCHOOLS” - EVOLUTION OF INFLUENCE OVER TIME - INFLUENCE OF PUBLICATION MEDIUM (PAPERS IN JOURNALS, PAPERS IN COLLECTIVE VOLUMES, MONOGRAPHS, THESES AND DISSERTATIONS) - INFLUENCE OF LANGUAGE OF PUBLICATION … D.Gile citations as indicators

  15. WHAT QUESTIONS CAN CITANAL HELP WITH?ASSESSING QUALITY? ARE CITATIONS A VALID INDICATOR FOR RESEARCH QUALITY? QUESTION: ARE THE MOST OFTEN CITED AUTHORS IN TS/IN OTHER DISCIPLINES THE ONES WHO DO THE BEST RESEARCH? D.Gile citations as indicators

  16. WHAT QUESTIONS CAN CITANAL HELP WITH?DEVELOPING THEIR USE - INTERACTIONS BETWEEN GROUPS (INCLUDING INTERDISCIPLINARITY) COLLECT INFORMATION ON CITED TEXT - FROM TS OR OTHER DISCIPLINES? Gile 2002/2006 TYPES OF CITATIONS - THEORIES - FINDINGS - RESEARCH METHODS - OPINIONS - TERMS… INDICATIONS ON QUALITATIVE ASPECTS OF TS RESEARCH EMPIRICAL/NON EMPIRICAL D.Gile citations as indicators

  17. GENERAL METHODOLOGICAL NOTE:THRESHOLDS ASSUME “EMPIRICAL”: FINDINGS, METHODOLOGY, REFERENCES TO EMPIRICAL STUDIES NON EMPIRICAL: TERMS, OPINIONS, IF 100% “EMPIRICAL” NO PROBLEM IF 1% “EMPIRICAL” NO PROBLEM IF 50% “EMPIRICAL”? IF 30% “EMPIRICAL”? NECESSARILY SUBJECTIVE/CONVENTIONAL THRESHOLDS THIS IS COMMON TO MANY EMPIRICAL PROJECTS WITH METRICS SUCH AS PAUSE LENGTHS, NUMBER OF E/O’S, etc. AT SOME TIME, SUBJECTIVE DECISIONS COME IN D.Gile citations as indicators

  18. GENERAL METHODOLOGICAL NOTE: CLASSIFICATIONS FOR MANY USES OF INDICATORS IN EMPIRICAL RESEARCH, CLASSIFICATION IS CENTRAL PRACTICAL ISSUE: NOT EVERY PHENOMENON ENOUNTERED IS CLASSIFIABLE: - MISSING INFORMATION • BORDERLINE CASES WHAT DOES ONE DO? CONVENTIONS, PERSONAL DECISIONS, INFORM READER PRACTICAL ASSIGNMENT TO GAIN FAMILIARITY WITH PROBLEMS ? D.Gile citations as indicators

  19. ASSIGNMENT (1) CHOOSE AT LEAST 3 SPECIFIC RESEARCH QUESTIONS FOR WHICH YOU BELIEVE CITATIONS ARE A GOOD INDICATOR Or THE EXPLORATORY QUESTION: WHAT CAN I FIND OUT ABOUT MY SAMPLE FROM CITATIONS? CHOOSE APPROPRIATE SAMPLE OF TEXTS TO TRY TO ANSWER THE QUESTIONS - At least 20 papers with a total of at least 200 citations if specific research questions - At least 5 papers with a total of at least 50 citations if your question is exploratory D.Gile citations as indicators

  20. ASSIGNMENT (2) • STUDY THE SAMPLE REPORT ALL THE CITATION DATA ON EXCEL FILE (SEE SUGGESTIONS LATER IN THIS PRESENTATION) • REPORT RESULTS AND ANSWER TO QUESTION(S) ON SEPARATE WORD FILE - REPORT CAREFULLY ALL METHODOLOGICAL PROBLEMS YOU ENCOUNTERED AND ALL METHODOLOGICAL LESSONS YOU LEARNED FROM THE EXERCISE D.Gile citations as indicators

  21. SUGGESTIONS FOR COLUMNS OF INFO ON DATA EXCEL FILE (1) 1. Citing author 2. (Affiliation) 3. Year of citing paper 4. References of citing paper 5. Page in which citation found 6. Cited author 7. (Affiliation of cited author) 8. (Discipline of cited author) 9. Year of cited reference 10.Cited reference D.Gile citations as indicators

  22. SUGGESTIONS FOR COLUMNS OF INFO ON DATA EXCEL FILE (2) 11. Type of medium (thesis, dissertation, article, article in collective volume, monograph, internet document…) 12. Language 13. Year of cited reference 14. Cited reference 15. Type of reference (opinion, definition of concept, term, finding, methodology, reference to empirical study, plain reference to publication, theory,…) 16. Other comments or key words D.Gile citations as indicators

More Related