kansas reflections

mindfulness in trading the markets, futbol, teaching, learning, leading, managing

Posts Tagged ‘research’

Reflections on critiquing the writing of others

Posted by Ken Long on November 3, 2009

Giving feedback about the paper is a way to show who you are and how much you care about the author.

Suppose, in your opinion,  the author has made a glaring error in logic or has not supported the thesis, or mischaracterized an opposing view, and because you are concerned about hurting their feelings, you don’t say anything.

How are you helping them? By letting their paper out into the world?

If you were right about the paper, and didn’t tell them, shame on you.

If you were wrong about the paper, that should emerge in the continued dialogue between professionals and now you have a chance to sharpen your own tool. You miss that chance if you don’t CRITIQUE THE PAPER, NOT THE PERSON.

If you comment on the paper without regard for the human who offered their vulnerability, their knowledge, their insights, THEMSELVES to you, try remembering to walk a mile in their shoes and ask yourself, before sending, have I been fair? have I been constructive? What is the tone of voice I used?

If you would say things anonymously about the paper in a double blind, but not to their face, that says more about you than about the paper you are critiquing.

Envision the paper as it leaves their hand and lands on a community table of knowledge for consideration. Focus on the paper on the table, not the person who offered it. The paper is not the person; restrict yourself to examining what has been offered. Don’t assume you know anything about their feelings or how they might take it. They have offered a piece of academic writing. Your duty is to evaluate it academically, while remembering there is a person on the other end, eventually.

The author has given us all a gift. Respect the gift by giving it your best critique: with support, with care, with your best work.  Respect the author for their gift and vulnerability. The critiques we offer are more important than anything we are likely to write on our own, and we will do a lot more of them than our own writing.

If you are an author, recognize the boundary between your Self and your paper. Be clear about what you are asking for when you offer it for review. If you want self-esteem more than honesty, you’ll get both, but not as you might want it.

Posted in Creativity, education, research | Tagged: , , , , , , , | 3 Comments »

Reflections on myself as an adult learner

Posted by Ken Long on October 16, 2009

Who am I as an adult  learner:

I am framing the answer within the context of my “Big 5” (Strelecky, 2007). The “Big 5” focus my thoughts about self, purpose, mission and values.  In Strelecky’s work, the Big 5 are 5 things you want to accomplish in your life. My “Big 5” are all states of being, roles that I want to live with the highest quality (arête). My Biog 5 are:  father, husband,  teacher, student, warrior,

 

Student:

Myers-Briggs Type Indicator:  ENTJ  I am getting closer to the “I” as I get older which moves me from “Leader” to “Scientist” in the typology.  My scores are very high on the NT domain, which gives me a global, theoretical perspective. I notice that I am always searching for the broadest generalizations that can be made from an incident, or the widest application of an idea. It doesn’t take much for me to go off on a tangent. I am least happy when bringing a project to a conclusion, as it feels stifling and disconnected from the dynamic world around me. Finality and endings are disturbing to me, and I dislike graduation ceremonies above all else. I am much more at home in the developmental and conceptual phases of any project. I get bored easily by data gathering and have learned to offload that task to others. I am a good project manager, as I have learned to build teams of various skills and aligning tasks with strengths.

 

Kolb Learning Style Indicator:

The Kolb LSI measures self-reported preferences along 2 dimensions:  Concrete experience-Abstract conceptualization and Active experimentation-Reflective observation.  These 2 dimensions reflect how we prefer to gather our information about the world and then how we prefer to make sense of it. The intersection of these 2 dimensions establishes 4 quadrants, and can be used as a way to describe a classroom population as well as individual learners. We use this model extensively at the Command & General Staff College, and I have become convinced of its practical uses when used within reason.

In this model,  I am classified as an “Assimilator”, which combines a preference for Reflective Observation and Abstract Conceptualization. This means I don’t need to spend too long “in the moment”, fully experiencing every nuance of the moment; I am always ready to begin reflecting on its dimensions, characteristics, descriptions and classifications. As an Abstract Conceptualist, I proceed to place experiences within my larger world view, as a particular example of a class of experiences. I spend little time in active experimentation to validate the data, once satisfied that it makes theoretical sense. 

                These preferences are helpful when approaching new material where the connection to theory is strong or explicit, because it satisfies my need to be situated in the world. I am comfortable with complexity and nuance and am  competent at brainstorming and imagining future scenarios.

The downside of my preferences is that I am prone to overlook deep subtleties in experiences especially if the situation is slow moving. The idea of sitting in a duck blind for hours waiting for birds is my idea of hell on earth. I am also prone to accept theoretical justification as truth and am willing to short change practical validation of new concepts simply because of the theoretical elegance.

                As a consequence of knowing this about myself, I find it necessary to do sitting meditations to work on my mindfulness and presence in the moment, to learn to appreciate the experience simply on its own merits, without a need to explain it or frame it as part of a larger construct. On  group projects I am careful to include pragtmatists and  naysayers who will insist on evidence and results from fair trials before we adopt policy changes.

                These strengths and weaknesses, and my accommodations to the limitations of my learning preferences are an integral part of my business success as an equity trader which puts a value of new ideas, but also on backtesting and forward risk management.

 

Brainmodepower typology: AVK, global. 

I am off the chart on the audio learning, and on the globalization scale.  I have now noticed that when I am really trying to concentrate on learning I do not look at the person talking, but need to doodle in order to free my ears to hear. Doodling helps me occupy my eyes and hands (visual and kinesthetic modes). This has been a problem for others in the past when they would say “Look at me and pay attention!”  when I was doing my best to pay attention.              

2 stories from combat on this topic which reinforces the power of the insight: On a night attack, wearing night vision goggles I had high explosive rounds land near me and “whiteout” my night vision goggles, and I lost my night vision for about 15 minutes: I was able to command my company though because I could hear what was going on via the radio and I had a sense of where things were based on noise, sounds, and the volume of fire. A few days later, in the daylight, I had a hand grenade land very near to me and I didn’t have my earplugs in. I was deafened for about an  hour before my hearing returned, and it was the most frightening experience I had ever had. I felt absolutely cut off from the world and was unable to command effectively. It was terrifying, even though I could see everyone around me and could consult a map.

 

Learning techniques:

I am a fast reader and I prefer to read in burst of 10-20 minutes, rendering my notes in visual, mindmapping form.  I will generally  develop detailed cognitive maps and turn them into slides as cues for recalling detail and cognitive structure. I take semi-structured notes on standard note-taking forms that I have developed over the years to suit my style. I will often color code the notes to make structure even more apparent. When I review notes from my Masters program (15 years ago), they make perfect sense to me and I can recall the circumstances of the classroom and the moment as if no time has passed.  This form of “chunking” supports my assimilating style. 

At any given moment I may be engaged in reading up to 20 books at a time in various locations, and I follow my mood or sense of urgency for picking up the next book to read. When I find myself drifting I stop and do something else until my attention is focused, rather than trying to force concentration.

 I can concentrate for hours at a time in reading if needed, but I prefer the shorter bursts when my mind is feeling especially sticky.  Learning to crate feelings of “sticky mind” is an essential part of my practice of sitting meditation, which Buddhists call “child’s mind”.

I will rarely read a book from cover to cover, preferring to read from top down and outside in, by examining the covers, introduction and forward, table of contents, index and references and chapter summaries first, and then come back to the book after 24 hours when that has had time to digest and become embedded. I will then skim chapters based on my interests, and finally skim the whole book. I have adapted this technique from  Mortimer Adler’s “How To Read A Book” (Adler, 1940) and it has helped me integrate a lot of material from a broad array of fields.

I am not very good in free form dialogues of material, preferring to hear structured presentations that reflect deep inquiry on the part of the presenter. Lectures are excellent for me as I can listen carefully, while doodling and seeming to daydream in my own personal comfortable space.  I enjoy writing and working on a topic while having a background lecture playing, trusting that if something interesting is being said that I will tune in to it. Some of my most creative work is done in this manner in the apparent cognitive dissonance set up by 2 different information streams. I am listening to a Teaching Company presentation on Chaos by Dr Stephen Strogatz as I write this.

My biggest problem as an adult learner is procrastination and time management, since I am always eager to read one more thing before generating my final conclusions. I also find it difficult to recast my theoretical framework of information once established and will generally try to find ways to accommodate pieces of my original insight in an evolving understanding. I try to delay taking final positions in order to gather more information for this reason.

I find it amusing that despite a strong rational component, and a structured approach to learning, that my decisionmaking and sensemaking is much more intuitive than rational.  I trust my instinct far more than my conscious mind. This is a habit perhaps ingrained into me from 15 years of  being an infantryman in combat and trusting my senses in dangerous situations. This habit of mind is so odd that it is even the subject of discussion among peers who know me well and wonder how I can be so rational and yet make instinctive, intuitive decisions.

 

Teacher:

I have been teaching in the Command & General Staff College for 8 years and have reinvented my whole approach to teaching as a result of the action research inquiry while attending CTU. While I acknowledge the need for competence at the data level I also have become much more aware of the importance of the social level of learning.  I no longer think that learning and education are like filling up a pail, but are rather like lighting a fire (to paraphrase  Yeats).

                I am trying to create an educational space in the classroom, in the college, and in my professional work that encourages and supports free inquiry, a commitment to truth and academic freedom, and both a respect for and a seeking out of diverse perspectives and points of view. As a teacher in the classroom  I try to model the behavior I seek from students, by the quality of my preparation, a concern for the learning and perspectives of others, and a willingness to be vulnerable in my ongoing search for knowledge. I am encouraging as many means of formal and informal feedback as possible to help students shape their own educational programs and outcomes. I encourage and support their inquiry in my classes and through support of their independent studies. I reach out to other colleges and programs to create networks of learners and to act as a catalyst for learning.

                I respect the action research construct of multiple ways of knowing (experiential, presentational, propositional, and practical) and acknowledge the learning that can happen through 1st person, 2d person and 3d person action research.

                I favor the connectivist learning school of thought being developed by George Siemens and Stephen Downes at University of Manitoba, as I believe it represents a realistic, sound, robust and challenging way of developing knowledge and practice to appreciate and thrive under conditions of uncertainty.  More at:  http://www.elearnspace.org/

                As part of my research I am looking carefully at how to add Voice to the environment by encouraging, supporting and promoting the diverse needs, intentions and inquiries of faculty, students and curriculum developers in a way that advocates a move away from an industrial age view of curriculum and towards one of connectivism and individuality. In this sense I have taken on an advocacy perspective that is values-based but which respects the perspectives of other members of the action research teams that make up the projects.

 

 

Father: 

My role as a father influences my role as a student. One of the important reasons for me to begin the doctoral program was to set  a personal example for my kids, who at ages 18, 15, 11 are getting to see their dad doing his homework and reading books every night as a priority.  My father set the same example for me as a kid as he went to night school to work his way up the engineering ladder from “shop rat “to full-fledged design engineer.  I’ve been trying to re-learn math and physics to be able to keep up with my son who is getting ready to go to college next year to be a physicist or an engineer, but just like in video games, I believe he has passed me for good. I am content to listen to him and get him the occasional book to feed his curiosity.

 

Husband: without my wife’s support I could not have dreamed of taking on the active role of student once more; in fact she finally told me to stop moping around and dreaming about it and just get it done. I need that boost from her to get moving at times. I want her to be proud of my work and my goals.

 

Warrior: 

I use Warrior in the eastern sense, as one who is called, by his dharma, to seek mastery of self first in order to protect the weak and promote justice and compassion in the world. This calling is well described in  Trungpa (1984). In this sense, my role as an adult learner is to focus on those things that I ought to be learning in order to improve my practice; to find worthy teachers and learn from them; to questions my own assumptions and preconceived knowledge in order to step outside what I already think I know and to follow my beliefs to their core to find the source.

Warrior learning also has a strong service component, and so the topics for inquiry, the choices for action research must satisfy the “so what” question, must be directed towards a virtuous end. For me, the choice to do action research within my college represents a way to do the right thing in support of my duty to country and soldiers whom I support. Action research’s methodology strongly supports these values, particularly when fellow inquirers are positioned as co-researchers.

 

References:

Adler, M. & Van Doren, C. (1940). How to read a book. New York: Simon & Schuster.

 

Strelecky, J. (2007). The big five for life: Leadership’s greatest secrets. New York: St Martins’ Press. 

Trungpa, C.  (1984).  Shambhala: Sacred Path of the Warrior. Boston: Shambala Publications.

Posted in education, family | Tagged: , , , , , , , , , | 2 Comments »

Management games for deep insight

Posted by Ken Long on October 6, 2009

Peter Checkland’s Soft Systems Methodology (SSM) describes the use of models to help us frame questions to ask of the world, and which help us become explicit about our world views, assumptions, frames of reference, theories of cause and effect, values, and desired outcomes.

Checkland, P. (2006) Learning for action: A short definitive account of soft systems methodology and its use for practitioners, teachers and students.  Chichester, England, Oxford Press

 We’ve developed a deceptively simple Force Mgt practical exercise in the form of a card game. The complete rule set is simple; takes 5 min to scan and understand. 

Rapid rule summary:

1. Students buy forces (5 cards) from a production table (a limited deck) and in each of 5 rounds,  deploy them into 5 regions to compete for Victory Points

2. Win:  first one to 51 victory points OR most points after 5 rounds

3. Game: lasts up to 5 rounds

4. Each round has 5 hands , each hand is worth Victory Points (VP)

5. Hand 1 is worth 6 VP, hand 2 is worth 5 VP etc…

6. Player 1 buys from the red deck, player 2 from the blue deck)

7. After you buy your 5 cards, you place 1 card face down in each region (hand)

8. Once all cards are placed,  cards are flipped over and you determine results

9. If your card wins the hand you get the victory points and keep you card; if you lose the hand, you get no victory points and lose your card. If it’s a tie, you keep your card and no one gets points.

10. Each player has an identical deck to buy from.

 It turns out that the development of strategy and then fielding an appropriate force really matters, AND there are distinct choices that are meaningful, available and feasible.

If you are interested, we’d like you to review the rules, and :

  • 1. Buy your first round of forces
  • 2. Deploy them into the 5 regions for turn 1.
  • 3. Send your “Round 1” move to long-kenneth@conus.army.mil, along with a short description of your strategy

 We are interested in examining the variety of forces and the strategy employed in round 1.  Do you, for example:

1. Buy 4 ea 10s and a Joker to kill any enemy aces and retain max budget  flexibility to see what he has remaining?

2. Buy aces early to get a lead on victory points and then protect them?

3. Buy Jacks to kill 10s while still preserving SOME budgetary flexibility?

4. How do you balance economy of force with winning victory points? (efficiency vs effectiveness)

5. Variations?

 And then tactically employing forces, do you:

1. Put aces against 6 and 5 victory point regions?

2. Put 10s against 6 and 5s to hunt aces?

3. Aim for maximum victory points each round?

4. Aim to capture 11 of the 20 available points each round? (ie bluff on 6 and 3, but try to win 5,4,2?)

 In the actual play of the game we’ll look for adaptability and learning, and how strategies change after teams have played each other a couple times etc. 

We’ve play tested it enough to know there is a rich source of insights available in the game and that it is simple to play. We’ll  play it with decks of cards in the classroom 

We prototyped the game in our Force Management elective and are satisfied that that we generate student interest and insight into broader questions of Army force management in an interesting way.

 Here are some student insights gleaned from our playtesting:

1.  Round 1 results dominate the rest of your strategic choices, so getting Round 1 is crucial.

2.  Round 1 strategies are dominated by uncertainty because you have no information about your opponent’s strategy or adaptive style yet.

3. You have to decide when you want to buy strength: early and aim for quick wins, or later after you have seen pieces of the opponents forces and strategy.

4. Forecasting your opponents moves is problematic and make this more like poker than chess or bridge.

5. Aces are like the FCS: dominating until low-cost alternatives found the weakness. It wasn’t unit Aces were developed that the 10s became meaningful, so be alert to deep flaws in complex technologies.

6. Kings are costly but dominate the field; An opponent with Kings drives you to buy Aces but make you vulnerable to 10s.

6. Jacks (J) are a low cost success strategy against 10s, but can be incrementally be defeated by other mid-weight forces.

8. The costs of transforming cards between rounds is significant but manageable and may lead to strategic advantage. Scenario: You buy Aces on the first round and are successful, opponent buys 10s to kill your aces in the second round, but you trade down to Kings which dominate, and which remain difficult to defeat in subsequent rounds.

9. Deciding where (in what regions) to selectively deploy strength

10. Tactical results can overcome strategic insights and strategic failures. Tacrtics can be game changing.

11. What if the enemy has different victory conditions? Price points? Has different rules?

12. What if new cards are introduced after the first rule set is established?

13. How much would you pay to see the opponents’ hands?

14. What if there are partial wins? Or more than 2 teams playing?

15. Simple games can be powerful learning strategies

 Conclusions: the game serves as a way to dramatize very clearly many of our force management challenges and is a useful way to create rapid, deep awareness of prime issues in this domain.

 Here are some insights from a dedicated gamer and management game modeler:

I suspect that for most people’s first play they are strongly influenced by a form of Confirmation Bias: the As are priced higher, therefore new players conduct their analysis from the assumption that As are more valuable. Depending on the goals of your concrete experience, that may be the best argument for keeping the current price structure. However, an ace of spades loses to seven cards, including four cheap ones, where a KH loses to only four cards that are both expensive and vulnerable — the KH is easily the strongest card in the deck.

I assume trade-ins are secret — in fact that for all practical purposes players are operating behind a screen during their setup phase — because knowing whether your opponent has made any trade-ins is very valuable information. You may want to specify that in the rules.

 Given the prevalence of 10s in everyone’s first turn strategies, it seems like the second-cheapest strategy is far more optimal than the cheapest — that is four tens and a jack of spades. That marginal $15 gives you a pretty good shot at a victory somewhere, and a decent chance of carrying more net capital forward.

 Here are a selection of previously submitted moves for  Round 1: (* = Joker)

Strategy 1
Region Cards Strategy:                  Cost:  102   Carry forward: 48
6 10h I’m trying to kill aces while creating and deploying one, but putting it where it is unlikely to run into an ace-killer unless the other guys is trying an ace-killer strategy like mine.  I’ve got cheap on the ace I bought, which is a risk that may not be worthwhile. I’m expecting to kill an ace in either 6 or 5, win 4 outright, and lose in 3 and 2.  Expected results are thus 9.5 points to me, 10.5 points to the bad guys, I will lose approx $35 worth of cards and kill approx $70 worth.  The enemy is expected to have spent rather more than me, so I will have more cash with which to restructure in light of what I find out. Cost: 102
5 10c
4 As
3 10d
2 10s
Strategy 2
Region Cards Strategy:                   Cost:  150   Carry forward: 0       
6 10s 10 is the ace killer on 6, then we try to overpower each successive category on the way down.  Assumes aces go to 6, which rapidly becomes a tail-chasing assumption. 
5 As
4 Ks
3 Qs
2 Js
Strategy 3
Region Cards Strategy:                   Cost:  123   Carry forward: 27      
6 Jh Hunting the ace-killers, retaining some flexibility, winning early points 
5 10h
4 Ah
3 Jc
2 *
Strategy  4
Region Cards Strategy:                   Cost:  150   Carry forward: 0     
6 Ah Maximum strength in every region 
5 Ac
4 Qs
3 *
2 *
Strategy  5
Region Cards Strategy:                   Cost:  145   Carry forward: 5  
6 Ah Maximum strength in main regions, try to hunt an ace and kill 10s; accept risk in small region 
5 Ac
4 10s
3 Js
2 *
Strategy  6
Region Cards Strategy:                   Cost:  149   Carry forward: 1     
6 10s Hunt aces and accept risk in regions 5,6, steal points with aces & J in regions 2,3,4 

Posted in Creativity, Markets, Planning, education, management, research | Tagged: , , , , , , , , , , , , , , , , , , , , , | 2 Comments »

A reflection on mixed methods research in adult education

Posted by Ken Long on September 21, 2009

Introduction

The Research Problem

The purpose of this paper is to offer one vision of developing a methodological theory of mixed methods research co-equal with that of quantitative and qualitative research. I use a case study of the US Army Command & General Staff College engaged in a redesign of its curriculum, its teaching practices and its design process itself in a period of revolutionary change while supporting a nation at war. I describe circumstances and worldviews in which I argue that only mixed methods research may be employed to simultaneously develop a deep appreciation of uncertainty, improve decision making through an appropriate gathering, mixing and analyzing of quantitative and qualitative data, and applying “learning in action” as a strategy to manage success.  I contrast the view of research  as a process of increasing knowledge for control  with a worldview of research as a learning-in-action that allows for deep appreciation of complexity but without the assertion that appreciation and research can lead to prescriptive measures of control.  I examine the merging feedback system of the CGSC curriculum redesign as a mixture of qualitative and quantitative data. The concept of “Voice” that emerges from the CGSC action research process will be described, along with a multi-phased, multi-year research plan that demonstrates the practical development of an interactive dynamic research plan that is also adaptive to interim and periodic results. The paper  reflects a pragmatic worldview as it focuses on practical outcomes inside an organization concerned with real-world results, but acknowledges the importance and utility of the other 3 worldviews described by Creswell (2007, p.6), namely advocacy/participatory, post-positivist and constructivist.

Waldrop (1992, 2008) described the emerging science of complexity in a rich description of the inter-disciplinary work developing at The Sante Fe Institute. Sixteen years later (Waldrop, 2008) he found that the pioneer days of complexity research had evolved into a rich diversity of programs in major and minor universities worldwide, with lines of business and cognitive domains each finding ways to apply the ideas of emergence, uncertainty and complexity in new and profound ways. What remained unchanged from the origins of the research were the questions of what next and so what and how much more is there and what does it mean to apply an appreciation of complexity to everyday problems and opportunities.  The field of education is only beginning to appreciate how complexity and uncertainty may change the dynamics, structure, content and practice of adult education (Siemens, 2004). Professions in particular will be challenged by educating for complexity, since deep, profound, and reliable bodies of knowledge are at the center of professional practice. Educators, themselves members of a profession, are examining what it means to educate, teach and instruct in light of an emerging awareness of complexity.

The US Army Command & General Staff College (CGSC) is a self-described “learning organization” (Senge, 2000), engaged in a revolutionary re-design of curriculum and teaching practice, with a mission to educate 1500 US Army Majors for uncertainty and complexity, while engaged in a global war on terror and in direct combat in Iraq and Afghanistan (Long, 2009). This provides an opportunity to examine reflective learners and practitioners in action (Schon, 1987) using mixed methods research and using multiple worldviews (Creswell, 2009; Creswell & Plano Clark, 2007).

Past Research on the Problem

Edmondson and McManus (2003) propose a structured approach to selecting research methods that fit the state of theory in a given field. Their 3 archetypes of the state of theory: nascent, intermediate and mature  are connected to qualitative, mixed methods, and quantitative research methods, resulting in an appropriate methodological fit that aims to meet the needs of researchers worldwide. Creswell(2009) offers a systematic approach to analyzing: researcher worldview, research purpose,  research questions, the state of theory, data collection, populations and situations to be studied, and data analysis in order to further refine the methodological fit  and better connect purpose with practice across all 3 methods. Creswell and Plano Clark (2007, p.8) offers a functional working description of the state of mixed methods research, which proceeds from a deep review of current field practice, establishes a superb framework for classifying current choices of mixed methods research design and the means by which methodological fit may be refined, but stops at the boundary of developing a deep theory of mixed methods of research.

Deficiencies in past research and Need for Mixed Methodology

Conventional professional education processes have been adapting at an increasingly frequent rate as a consequence of Army senior leader directives and direct field feedback. The adaptive processes and decisions to date have been single issue, single iteration problem solving exercises inherited from an environment in which incremental change was the norm and most appropriate. These processes are less and less suitable as rates of required change increase and the relevance of existing processes and curriculum are increasingly called into question (Long, 2009)

Audience

The audience for this research include: staff, faculty and students of CGSC; educators of military professional schools; curriculum developers in graduate schools and organizations engaged in preparing leaders for uncertainty; scholar-practitioners of mixed methods looking to adapt practical field methods of for mixing qualitative and quantitative data; scholars examining the deep theory of the methodology of mixed methods.

Purpose

Purpose of the study, and reasons for a mixed methods study

The purpose of this study is to examine the CGSC curriculum redesign project and the emerging feedback system that guides design decisionmaking, which incorporates both quantitative and qualitative data. The project can be described as intermediate theory in the Edmondson  and McManus ontology, and therefore suitable for mixed methods research since I am introducing an emerging concept (“Voice”), focusing on the exploration of theoretical propositions (the theory of mixed methods), the availability of sets of rich theory that inform the research (adult learning, decisionmaking, complexity, design, learning organizations, narrative inquiry and action research), and incorporating multiple data types and analysis (Edmondson& McManus, 2007, p.1165).

Research Questions and Hypotheses

Quantitative Questions

H1: Student satisfaction measured on the Noel-Levin Adult Learner Satisfaction Survey is not different than their reported overall satisfaction

H2: Student education priorities measured on the Noel-Levin Adult Learner Satisfaction Survey are not different than those of faculty and college leadership as measured on the same instruments

H3: Student education priorities measured on the Noel-Levin Adult Learner Satisfaction Survey do not vary through time in the course of the academic year

H4: Student education priorities measured on the Noel-Levin Adult Learner Satisfaction Survey do not vary after graduation and reassignment to field units

H5: Student satisfaction measured on the Noel-Levin Adult Learner Satisfaction Survey do not vary from satisfaction as measured by existing CGSC Quality Assurance  surveys

Qualitative Questions

What are the dominant and subordinate narratives that emerge from focus group discussions on educational priorities and practice and environment within CGSC?

How does the curriculum design decisionmaking process respond to similarities and differences in narratives that emerge from groups of students, faculty and senior leaders?

Describe the development, emergence of the construct of “Voice” from the CGSC PAR cycles, and how this prototype construct is evolving and being applied by various sub-groups within and associated with CGSC, by applying various interpretive methods of the narrative inquiry tradition.

Mixed Methods Questions

1. To what extent are qualitative insights generated from PAR cycles, focus groups, and individual interviews supported by quantitative data generated from surveys and actual use data of digital communication and collaboration mediums?

2. How are various organizational narratives constructed by sub-groups within the CGSC curriculum design process in order to make sense of quantitative data?

3.  What insights are offered by the application of  various narrative inquiry traditions? Which traditions are favored or overlooked or rejected by curriculum design decisionmakers?

4. What happens within CGSC when students and faculty are given opportunities to exercise “Voice”?

Philosophical Foundations for Mixed Methods Research

Quantitative research literature review

Student satisfaction surveys built on consumer theory (Watkins, 2009) are broadly applied in colleges and universities, and  treat students as  free-willed individuals that choose between alternatives for an educational institution and particular fields of study. They are seen as rational actors with definite expectations about what they want in their educational experience, and that satisfaction occurs when their expectations are met or exceeded. Smart, Feldman and Ethington (2006) note a decline in the attention being paid to the attitudes and behaviors of faculties, administrators, and the college and university environments as contributors to student success.(p.2.). These insights are related to the “college impact” model of student success. Applying the Noel-Levitz  Adult Student Priorities Survey leverages a robust, nationally recognized, validated research instrument whose dimensions reflect the areas of importance emerging from the CGSC Participatory action research  (PAT) study (Long, 2009) and enables  quantitative research into the existing database of historical satisfaction measures currently applied in the college’s curriculum design process.

Qualitative research literature review

The James, Milenkiewicz and Buchnam (2008) application of Participatory Action Research (PAR) develops measureable action steps that can lead to  revolutionary transformations within educational institutions.  The use of measureable qualitative and quantitative data gives power and legitimacy to the insights it generates inside an organization that values rigor and validity, while respecting the intuitive insights of qualitative research.  Prasad describes many techniques of Narrative Inquiry that offer many techniques for interpreting and making sense of qualitative and quantitative data.  Reason & Bradbury, (2008) and Clandinin, (2007), describe these disciplines and crafts of action research and narrative inquiry as having a relatively mature foundation of theory and best practices, with enough variation between sub-disciplines as to create real and significant choices for researchers . Various methodologies in each discipline can be characterized according to their own logic that connects their particular world view (Creswell, 2009), ontology, research technique, data requirements, classification and analysis protocols, and strategies for sense-making of the results of inquiry. The combination of PAR  and narrative inquiry offer a robust set of strategies for generating insightful qualitative data with connections to quantitative data sets, which make them especially useful and practical to mixed methods researchers.

Mixed methods research literature review

Creswell and Plano Clark (2007) provide a broad yet detailed overview of the current state of the art of mixed methods research. They offer a working definition of the field derived from a survey of practice which proceeds from a deep understanding of high quality methods of practice, through choices of design and point to potentials  for the development of deep methodological theory. They offer mixed methods as an appropriate research strategy as a way to improve on the use of either qualitative or quantitative research alone.  In their view, mixed methods are more comprehensive, can answer more  types questions, encourages collaboration  and the deliberate incorporation of more than 1 worldview and is especially well suited for situations where practicality and pragmatism are prized (Creswell & Plano Clark 2007, p8-11).

Methods

A definition of mixed methods research

Creswell and Plano Clark (2007, p.5) define mixed methods research in the following way:

“As a method, it focuses on collecting, analyzing and mixing both quantitative and qualitative data in a single study or series of studies. Its central premise is that the use of quantitative and qualitative approaches in combination provides a better understanding of research problems than either approach alone”.

The type of design used and its definition

In this section I will briefly describe the theoretical shortcoming of treating mixed methods merely as a practical solution to improving upon either the qualitative or quantitative approach alone, and why a broader and deeper theory of mixed methods is appropriate for developing deep appreciation of complexity and uncertainty. I will briefly describe two different designs that would pass the test of the Creswell and Plano Clark ontology of mixed methods designs. With appropriate development, either would be approved for research within CGSC.

Comparison table  (adapted from Creswell & Plano Clark (2007)

Design 1 Design 2
Choices Explanatory Exploratory
Theoretical Description A 2 phase  design, where qual helps explain or build upon initial quan results (p.71) 2 phase design where qual results help develop or inform 2d phase quan inquiry
Description of application to CGSC Round 1: the Noel-Levitz Adult Learner Satisfaction Survey is applied to a population, and results are tabulated, analyzed, compared against national  graduate student norms and in  a time series from the beginning, midpoint and endpoint of the academic year. Insights are developed

 

Round 2:  A series of focus groups and individual interviews are used to develop qualitatitive insights to  make sense of the quantitative findings

Round 1: a  set of participatory action research cycles identify areas of pressing concern to leaders, faculty and students within the college. A grounded theory is developed and constructs are defined by the community of practice, informed by theory from PAR outsiders.

 

Round 2: A quant survey is developed to explore deeply into issues and constructs developed by the PAR teams to cross check for validity, to confirm or deny, to support or modify the emerging grounded theory and provide the basis for future inquiries as selected by PAR teams. (Note: this is s summary of the actual process used at CGSC as the basis for this case study. The various tangents deriving from the initial rounds of inquiry generated my epistemological concerns with the pragmatic assumptions of mixed emthods)

Design notation QUAN->QUAL QUAL->QUAN
Justification Needs qual to help explain significant, , non-significant, outlier or surprising quant results Exploration is needed because:

 

1. no existing instrument

2. unknown variables

3. immature theory or framework

Well suited for exploring a phenomenon or when researcher wants to generalize to other populations, test emerging theory or classifications (p. 75)

Variants 1. Follow-up explanations (quan results, insights need additional explanation)

 

2. Participant selection  (where a sampling of representative outliers are selected for follow-on inquiry)

1. instrument development model

 

2. taxonomy development model

Strengths Straightforward implementation

 

Feasible for single researcher

2 section report of results

Supports both single and multiple phase studies

Appealing to quan researchers

1. easy to design, describe, implement and report

 

2. although initial emphasis is on qual, the quan phase makes it easier to appeal to quan audience

3. both variants supports multiphase studies well

Challenges 1. Time consuming

 

2. Decisions on which individuals to use by phase w/justification

3. Difficulties with IRBs

4. Deciding which results to explore

5. Specifying criteria for follow-on inquiry (before or after results?)

1. time consuming

 

2. difficult to specify phase 2 construct for IRB prior to phase 1 results

3. deciding up front which individuals to use in phase 2

4. which data to use in phase 2 instrument

5. deciding relevancy of phase 1 results for phase 2 taxonomy

Timing 2 phase sequential model 2 phase sequential model
Weighting I think the QUAL(quan) model is more likely. This design relies on at least a mature enough state of theory to allow for initial quan inquiry, but we are more concerned with the interpretation and application of insights than in model or theoretical validation the equal weighted choice is more logical; the desired outcome is an improvement to state of theory (quan) by either a better instrument or by an improved taxonomy (ontology). Yet the reliance on initial qual inquiry as a guide makes it at least co-equal to quan.
Mixing the data Either Merging or Connecting is more likely than embedding.  Embedding implies a single phase, whereas this is defined as a 2phase design.  The improved explanation of initial quan findings is how the design could be “connected”. If the interpretation or meaning making is intended to create “rich description” then either variant of merging is logical Connecting is by far the most logical design choice, as the 2 phases are explicitly linked; quan follows qual and the connection is either an instrument or a taxonomy.

 

There is a distinct “manufacturing or processing” aspect to this design, which does not seek to produce a rich description that is a blend, but rather produces a better quan framework as guided by the initial qual inquiry

Diagrams:

Research model 1: Explanatory:

Research model 2: Exploratory:

Analyzing the data:

In both models of mixed methods design the quantitative data would be subjected to power analysis, tests for relationship and causality.  The quantitative hypotheses are framed in the form of null hypotheses in order to determine if there were differences that could be attributed to a difference in instruments and what they are measuring (existing survey vs the Noel-Levitz survey); through time series tests to see if there is a treatment effect, and with the samples subjected to control variables to examine the effects of demographic, career experience, educational goals and faculty specific effects on the measures of satisfaction and importance.

Qualitative data would be subjected to thematic analysis according to the practices of the grounded theory, narrative inquiry and PAR traditions/  Narrative inquiry traditions are especially important here as outlined in Boje (2001).

Analyzing the mixed data would be drivcefn by the specific design selected as noted above.

Theoretical analysis of the consequences of choice in mixed methods design

Both designs would be interpretable as providing a deeper insight and understanding than a study restricted  to either of their individual qualitative or quantitative components. The functional definition of mixed methods (Creswell & Plano Clark, 2007, p.5) would lead decisionmakers, particularly of the pragmatic worldview, to ‘receive the wisdom of experts” and seek to straightforwardly apply the insights based on a justifiable belief that they now knew more about what was going on, and had in some fashion reduced the amount of uncertainty about the world around them. My central argument is that there are situations so complex and uncertain that no amount of research and conclusions drawn from best practices of the traditions of both quantitative and qualitative inquiry, and the best practices emerging from mixed methods as described by Creswell and Plano Clark. In fact my use of the word “situation” in the preceding sentence, is an implied assumption that there is such a thing in the real world as a definable “situation” or problem set  which may be bounded and contained by a problem solving, decisionmaking entity. While this construct is the basis for the post positivist tradition, which has endlessly proven its utility in countless settings, it is normal for pragmatists to conflate utility with reality.

It can be argued that there is could be a tacit agreement between constructionists and post-positivists to allow each other the primacy of method and interpretation based on typical problems, and indeed much work is being done to increase cross-discipline understanding, cooperation and integration. The common assumption between these two worldviews is that the end product of such effort is a measureable increase in our knowledge of the world as it is, from which we may exert more control and prediction by having reduced the amount of uncertainty by some amount. This tacit shared assumption I submit is expressed through the research practice of pragmatists who are “naturally” drawn to the mixed methods designs and practices described so well by Creswell and Plano Clark. Given the fertile and as yet only partially explored areas best suited for mixed method research it would be natural for the deeper philosophical theory or theories of mixed methods to be postponed, much as Smart, Feldman and Ethington (2006) found a willingness for researchers to revert to their preferred and more easily measured  research domains and begin to neglect the messy and challenging issues of environmental factors affecting student success. One is reminded of the story of the man who’d lost his key in a dark alley but was searching for it under the streetlight because the light was better there.

I am arguing that there are situations where even mixed methods are properly and rigorously applied, and interpreted in best professional practice, that the insights may serve only to help decision-makers appreciate the vastness of what they do not understand, and better act within an uncertain environment, humble in their ignorance, yet moved to action from values and on the basis of principles informed by the best practice of inquiry.

It is my contention that in those situations described so aptly as “wicked problems” by Rittel & Webber, (1973) that a  deep theory of mixed methods may be developed that is co-equal to that of qualitative and quantitative methods. I argue that mixed methods not only are useful in solving less-than-wicked problems, as described by Creswell and Plano Clark,  but most appropriate to engage with uncertainty and complexity for the express purpose of appreciating deeply the current situation. The deep theory of mixed methods I anticipate would require explicit inclusion of all 4 world views, since there is no a priori basis for excluding any of the 4. I thinkit quite likely that a reasonable assumption of a deep theory of mixed methods in fact could require an explicit inclusion of the best practices of each world view in some fashion, details to be determined, of course.

The shift in epistemological  perspective seems important to me,  and which should be developed in tandem to the directions for improvements in design and pure method described by Creswell and Plano Clark.  Checkland’s application of soft systems methodology,  artfully describes  “learning towards success” in a satisfying way (Checkland & Poulter, 2006).

The best expression of the theoretical stance towards irreducible complexity intersecting the human need for the state of nature or through any objective criteria (Boje, 2001) and in the work of Hayden White (1987) concerning the relationship between narrative discourse and the historical representation. These insights are causing me to reflect deeply on my own essentially pragmatic worldview and its underlying assumptions, and lead me inevitably back to the proposition that we need the methodological theory of mixed methods developed simultaneously with that of its design choices and specific methods.

References

Argyris, C.  (2008). Teaching smart people how to learn. Boston, Harvard Business Press.

Astin, A. (1999) Student involvement: A developmental theory for higher  education.  Journal of College Student Development (Sep/Oct, 19990 (Vol 40, No 5)

Beltyukova, S. & Fox, C. (2002) Student satisfaction as a measure of student development: Towards a universal metric.  Journal of College Student  Development (Mar/Apr 2002)

Checkland, P. & Poulter, J. (2006). Learning for action: A short definitive account of soft systems methodology and its use for practitioners, teachers, and students. West Sussex, England,  John Wiley & Sons Ltd.

Clark, M. & Rossiter, M.  (2008). Narrative learning in adulthood.  In S. Merriam (Ed) Third Update On Adult Learning Theory (pp 61-70). San Francisco,  Josey-Bass.

Creswell, J. (2009). Research design: Qualitative, quantitative and mixed methods approaches (Third ed.). Thousand Oaks, CA: Sage Publications, Inc.

Dorner, D. (1996). The logic of failure: Recognizing and avoiding error in complex situations. New York, Metropolitan Books.

Edmondson, A., and McManus, S. (2007). Methodological fit in management field research. Academy of Management review 32: 1159-1176

Fenwick, T.  (2008). Workplace learning: Emerging trends and new perspectives.  In S. Merriam (Ed) Third Update On Adult Learning Theory (pp 17-28). San Francisco,  Josey-Bass.

James, E. A., Milenkiewicz, M., Buchnam, A., (2008).  Participatory action research: Data driven decision making for school leadership. Thousand Oaks, CA: Sage Publications

Kearsley, G. (1997). The Virtual Professor: A Personal Case Study. Retrieved from http://home.sprynet.com/~gkearsley/virtual.htm

Lincoln, Y. (1983) Expectancy theory as a predictor of grade-point averages, satisfaction and participation in the college environment.  Annual meeting paper, Association for the Study of Higher Education, 1983.

Long, K. (2009). Participatory Action Research pilot study notes. Ft Leavenworth, KS: CGSC (unpublished).

March, J. (1994). A primer on decision making: How decisions happen. New York, The Free Press (Simon & Schuster Inc).

Prasad, P. (2005). Crafting qualitative research: Working in the postpositivist traditions. Armonk, New York, M. E. Sharpe.

Rittel, H., and M. Webber (1973). Dilemmas in a general theory of planning.   Policy Sciences, Vol. 4, pp 155-169.

Schon, D. (1987). Educating the reflective practitioner. San Francisco, Jossey-Bass.

Senge, P., McCabe,N., Lucas, T., & Kleiner, A. (2000). Schools that learn: A fifth discipline fieldbook for eductaors, parents and everyone who cares about education. New York, Doubleday.

Siemens, G., (2005). Connectivism: A learning theory for the digital age.  International Journal of Instructional Technology and Distance Learning 2 (10), 2005.

Smart, J., Feldman, K., & Ethington, C. (2006). Holland’s theory and patterns of college student success. Commissioned report for the national symposium on postsecondary student success: Spearheading a dialog on student success

Taylor, K. & Lamoreaux, A.  (2008). Teaching with the brain in mind.  In S. Merriam (Ed) Third Update On Adult Learning Theory (pp 49-60). San Francisco,  Josey-Bass.

TRADOC Pam 525-5-500. (2008).  Commander’s Appreciation and Campaign Design.

Trochim, W. Research methods knowledge base (2006). Types of designs.  Retrieved from http://www.socialresearchmethods.net/kb/desdes.php

Waldrop, M. (1992,2008). Complexity: The emerging science at the edge of order and chaos. New York, Simon & Schuster, Inc.

Watkins, T, (2009). Consumer theory in economics. San Jose State University Economics Department website. Retrieved from http://www.sjsu.edu/faculty/watkins/constheo.htm.

White, H.  (1987). The content of the form: narrative discourse and historical representation. Baltimore, The Johns Hopkins University Press.

Posted in education, research | Tagged: , , , , , , , , , , , , , , , , , , | 4 Comments »

The Many Distinctions of Narrative Inquiry

Posted by Ken Long on August 2, 2009

While conducting a methodological review of narrative inquiry as a technique to create meaning from heaps of data, both qualitative and quantitative it quickly became apparent that narrative inquiry is a much broader and deeper and diversified field than I originally thought. It is such a rich area that it deserves some detail research in order for me to find the appropriate method to apply to my particular research situation. A cursory review of the literature discovers at least 20 different techniques with various distinctions that are available to the qualitative researcher.

Here is a partial listing of just some of the different strategies and perspectives that a narrative inquiry can offer the qualitative researcher:

*   recording the experience of living the story as it occurs
*   interviewing the storyteller to compare their story with factual data
*   Language experts who analyze the forms and modes of the storytelling independent of content
*   Constructivists looking for methods of sense making and storytelling
*   deconstructionists looking to uncover the textual meaning
*   cultural storytellers looking to integrate storytelling in the larger culture
*   organizational developers comparing the nature of culture to the nature and power of storytelling
*   dramatists interested in the performance aspects and knowledge creation aspects of the act of storytelling itself
*   ethnographers looking to create a rich description of the storytelling environment
*   the Boje school of antenarrative which questions the validity of storytelling and grand narrative as a truth-hiding process
*   pragmatists looking to assess the effect of a change in storytelling style on various dimensions of performance
*   mixed methodists looking to use narrative to infer meaning from quantitative data
*   Post positivists looking to validate storytelling through confirmation by quantitative data

Posted in education | Tagged: , | 1 Comment »

The variety of perspectives in narrative inquiry

Posted by Ken Long on August 2, 2009

when I was reviewing my research questions for the qualitative portion of my mixed methods project, I became intrigued by the idea of narrative inquiry as a process to help me make meaning out of the reams of data that I had been accumulating from various cycles of participatory action research with students, faculty, curriculum developers, administrators and senior college leadership in a variety of settings. It looked to me to be the connecting link between undifferentiated data and supportable, meaningful conclusions arrived at in a rigorous, disciplined, informed way. So I started to dig a little deeper into the methodology.

What I found was a very broad and extended family tree of methodologies that were all calling themselves part of narrative inquiry but whose individual distinctions were so great as to make a deeper study necessary to pick the one best suited for me and my research. What I initially assumed would be a simple inquiry into storytelling and meaning making has become a detailed review of theory and may actually cause me to produce a cognitive map of the methodological domain so that I can be satisfied I’ve chosen my method or methods correctly when it comes to interpreting my research data.

After just a brief review of the methodological literature in several collections of books that provide frameworks for doctoral research students I found that the categorization of narrative inquiry within the field of qualitative research varies by author and to a much larger extent than I originally expected. In one such book, Prasad(2005) reports on 19 different varieties of narrative inquiry and takes a sampling of five to give a representative overview of the method. Even the variation between these five show quite a distinctive array of technique, although they do tend to share deconstructionists worldview. Prasad traits narrative inquiry as perhaps the highest level descriptor in the qualitative research hierarchy. Rudestam and Newton (2007) however, see it as simply one of the main branches of qualitative research, coequal with at least six other subcategories. In the Handbook of Narrative Inquiry, Clandinin describes narrative inquiry more like a continence filled with a rich diversity of loosely affiliated Tribes (my dramatization) which share related elements of philosophy and worldview, but in a different mixed depending on the to being compared.

It is clear to me that narrative inquiry is a rich mine full of valuable or to be discovered, and works into an even more useful tool, and thus my interest in it.

In a later post I will describe some of the most prominent distinctions between subdisciplines of narrative inquiry.

Posted in PAR journal | Tagged: , , | Leave a Comment »

Traders Roundtable: Maximum Compounded Return Versus Fear of Drawdown

Posted by Ken Long on August 2, 2009

In a recent traders roundtable discussion on the subject of balancing maximum gain with risk management the question was posed how to reconcile these two issues. A few definitions for the purposes of this essay are in order.

First let’s consider maximum compounded return to be defined as a system that is traded at every viable opportunity at the maximum level of acceptable risk with profits that are rolled into the total portfolio and with maximum acceptable risk in leveraging the markets money. This definition allows us to stay within natural risk limits but operate the machine as close to the red line of sustainable performance as possible.

Now let’s consider the fear of drawdown to be defined as the natural human response to operating power tools or heavy equipment in a dangerous setting as close to the edge as possible with full recognition of the consequences implied of potentially catastrophic failure. The human mind is capable of creating shades of distinction between various perceived levels of risk and these degrees of perception vary by individual. Some people are fully physically capable of walking within 5 feet of the edge of the balcony where as others can go right up to the edge and peer over and still remain in full normal natural control of their reactions. For each person though there is a line beyond which you judge that you are in dangerous territory in that extraordinary measures of protection and care must be taken because of the increased risk.

I am certain that there is a lot of evolutionary biology involved inside our brains which had to face this exact challenge on every front in prehistoric times or more technically, in the era of evolutionary adaptation. Even the most primitive caveman was certainly capable of appreciating the pay off of killing a mammoth for the future of survival needs of the tribe. And yet that same caveman was fully aware of the danger to himself that he took by stocking the huge beast. Sharpened by fear and hunger in the visions of a plentiful tomorrow, each caveman had to reconcile distention in some fashion and the successful ones passed on their adaptations to countless generations. This manifests itself in modern times in any place where risk and reward are brought together and we asked humans to make a balanced trade-off decision.

In every significant situation, and by significant I mean where impactful money is being rest for an impactful reward, the brain is flooded with chemicals which trigger flight or fight responses and invoke millions of years of stored up collective unconsciousness which shape and color our decisions and effectiveness of implementing those decisions. Trading is no different.

My sense is that for long-term safety and survival and given a trading system that generates a significant number of opportunities that long-term survival should drive us towards finding the minimum level of risk with which to trade to meet our specific financial goals. For those whose goal is expressed as maximum compounded rate of return, I suggest they are more likely than not to push it past the red line and come crashing down.

Without an appreciation of the real cost and friction associated with long-term trading in multiple market conditions that require constant adaptation is too easy to extrapolate the results of a few successful trades into next taxation that is far from being achievable. These false expectations are more likely than not to add to the level of stress and further degrade performance. My sense is that minimum risk levels rather than maximum risk levels are appropriate place to begin your inquiry into long-term trading as a potential career.

Therefore, I strongly suggest that you start with ways to use part-time small position sizing trading and learn to supplement your current income at your current standard of living and proceed in small stages, and getting larger only when supported by the evidence of long-term performance. It’s an incremental approach, it does not generate lottery size winds, but it will keep you in the game while you’re learning and keep your feet planted firmly on the ground.

Posted in trading | Tagged: , , , , , , , , , | 1 Comment »

Testing the Word 2007 Blog publishing template

Posted by Ken Long on August 2, 2009

This is to see how things come through when written in Word 2007 and pushed to the blog site.

Here is a sample picture:

 

Am wondering how tags and categories are handled?

Posted in PAR journal | Tagged: , , , , , , , , , | Leave a Comment »

Traders roundtable discussion: the issue of optimization in research

Posted by Ken Long on August 2, 2009

We were having a traders’ roundtable discussion on the topic of researching potential trading systems and the issue of optimization came up. This is a very important topic for traders who want to apply a systematic approach to trading markets. Here are some of the highlights of that discussion for you to consider as you prepare your trading strategies.

Typically when we think of optimization for a trading system we are looking at a process that incorporates multiple variables, parameters with different settings possible and perhaps a number of market filters or conditions which taken together with an exit strategy give us a multitude of ways to trade a particular concept or idea.

It normally begins with an idea the trader has based on insights fromtheory or from reflective practice where he believes the system gives a persistent advantage compared to the average market return  of simply buying and holding. It is also possible however that the edge may come from a brute force data mining operation that finds a statistical edge in some combination of market conditions. In other words, the insight comes from the result of massive computations and not from an intuitive or academic insight.

In either case , what we have is a system of multiple components, each of which can vary, and on an initial pass through with middle-of-the-road parameter settings we find a persistent edge in multiple markets with a statistical significance. Human nature being what it is, we would want to start testing different parameter settings for each component in order to determine the best mix for the most robust return and to find which of the parameters seem to have the most power when it comes to influencing the results. In statistics, this general approach is called factor analysis or principal component analysis.

In theory, you would want to find the absolute maximum return by finding the absolute optimum setting for each of the possible parameters and then take that into the market to begin trading. Taken to an extreme, this can produce a phenomenon of curve fitting or over optimization. What you can end up with is a system that would be perfect for the unique set of data conditions of the test. The problem of course is that the future may not ever show you that same data set again in your over optimize system will under perform much to your surprise.

The usual response to this phenomenon is to conduct testing with out of sample data. In other words designing the system on one data set and refining it to a certain degree and then testing it on a completely new set of market data to see if the edge persists. If you were systems development practice find you always discarding your systems after the out of sample test, it probably occurs as a result of over optimization.

In practice then, we want to find the trade-off between robustness of performance in multiple market conditions with out of sample testing that yields a persistent advantage in multiple market conditions but without trying to overturn the system for ideal conditions.

A way to keep this systematic approach in tune is to continue to monitor performance in a feed-forward approach that examines actual trading results to see how the performance results compare to the test and confirmation performance curves.

The bottom line: the more you rely on automatic trading systems, the more important your research and validation process becomes since you will not be using inexperienced traders override protocol to keep you from going off the deep end.

Posted in Markets, Teaching, Uncertainty, education, management, research | Tagged: , , , , , , , , , | Leave a Comment »

The future of education in a net-centric world

Posted by Ken Long on July 27, 2009

reinventinglife.org 

Surveys of CEOs and futurists alike all agree that the future is calling to us with a need for open systems, open thinking, collaborative organizations, workplaces and attitudes.  Net-centric  democratic education,  public and private movements for social justice, moral and honorable business enterprises,  and cross cultural communications all place a premium on skills, tools, and attitudes that encourage and multiply the network effect.

Doctoral research should reflect this same willingness to push boundaries, and encourage the use of collaborative teams of researchers and practitioners to explore collaborative theory and practice. If collaboration works in practice, there needs to be a theory of collaborative action, and the research in support of that practice should reflect the same principles as the work itself.

Like all new movements, there is a bootstrap problem when you try to go beyond conception into first action.  How do you create the initial conditions to generate the spark of collaboration which can ignite the passion and energy of research teams? The words sound good, but we need a place to try it on for size; We need supportive yet challenging places to discover, invent , develop and propagate the  best practices and techniques of collaborative knowledge creation.  We need places to test our theories, tell our stories and plan for the next cycle.

Dr Alana James’ growing community of practice at reinventinglife.org, centered on the principles and techniques of Participatory Action Research, is just such a node. It serves as an incubator for researchers and doctoral students seeking to develop the net-centric research effect. It goes beyond dry academic reports of practice, by being a live and vibrant laboratory for discovering, applying and reporting on the “theory-in-use” of open education.

It is a combination of challenging questions, a tool repository for new applications of digital learning and research, and is creating a circle of multiple competencies that will support  spin-off collaborations in a variety of settings that will propagate waves of change throughout the network like waves in a pond.

Dr James’ leadership in this area is paving a way for new approaches to research and practice and is exactly the kind of practice which needs support and recognition. She has the vision to imagine revolutionary positive changes in the world around her, the moral courage to invest in encouraging the Voices of students and co-researchers as we work to define and realize that future, and the skills to lead and encourage that change.

Posted in Creativity, PAR journal, Teaching, education, leadership, management, research | Tagged: , , , , , , , , , | 2 Comments »