Reading: E-learning, Chapter 3

isd581
#1

Read the Chapter. Write your response by Thursday. Reply to at least 2 people’s posts. At least one of your responses should be written after Thursday when everyone has had a chance to post. Ideally, you’ll participate on at least three different days.

What to Look for

Which research question (p. 52) do you think that you might be most interested in studying and why? What tools or methods might you use?

#2

To begin, Clark & Mayer (2011) stated that their focus is on “instructional effectiveness” (pg. 51). Furthermore, they define this as “identifying instructional methods or features that have been shown to improve learning.” (Clark & Mayer, 2011 pg. 51). As instructional designers, we are all called to do the same thing. We are in pursuit of better learning. For me, this means that I would have to focus on researching what works. I am driven to know what makes a learner learn to the best of his or her ability.

Currently, at my job, we are exploring the best practices for teaching grammar and writing. We think that our students are not transferring the skills they are learning to all of their classes. I have made some recommendations, but I should encourage some scholarly research on the topic. I am luck because I have access to a great database here at USA. Before I was enrolled here, I had to rely on Google Scholar or the public library. Unfortunately, these sources are not the most fruitful. Many times, I found that they wanted me to pay for articles from Google Scholar. I might find a snippet of a book, but not be able to access the whole book. Usually, the public library contained mostly pop culture information.

Although, I am sure experimental research is the best way to determine the best practice This will be covered in Research and Evaluation for any IDDers who have not take those classes yet. it is also costly and time consuming. Many times a “normal” person cannot just whip up an experimental study.

I think an important step to any method is a literature review. Explore what other people have already done. This way, you can avoid some mistakes. Clark and Mayer (2011) suggested “focusing on situations that are like” (pg. 55) your own, have good research methods, and relays the appropriate statistical information. Another useful tool might be finding a person who is doing the behavior you want to see. For online instruction, look at all the professors. See who is getting good ratings and results; then, examine what they are doing. You can observe and examine what is working for someone who is already practicing and has seen success.

2 Likes
#3

I have to agree with your point. Clark & Mayer (2011) discussed three methods to research that are used to demonstrate the effectiveness of instruction. As a special educator, I am interested in “what works?”, “how does it work? and “Why it did not work?” Therefore, experimental comparisons might be the method that I will need to use. I would like to try to work with experimental group and a control group. However, as a “normal” person, this experimental study has too many design issues: sufficient learners for both experimental and control groups, was the learning measure sensitive enough? etc.

2 Likes
#4

Clark & Mayer (2011) discussed three methods to research that are used to demonstrate the effectiveness of instruction. As a special educator, I am interested in “what works?”, “how does it work? and “Why it did not work?”

Therefore, experimental comparisons might be the method that I will need to use. I would like to try to work with experimental group and a control group. I could compare test performance of a group of students who learned to use self-checklist of the steps needed for solving a long division versus a group who solve the long division without the check list.

However, as I read the chapter, I realized a few things: 1) I may not have enough learners in each group, 2) I have to make sure that the treatment and control group are different enough from each other, so I could have an adequate treatment implementation, and most importantly, 3) I need to make sure I plan for confounding variables.

In addition, I want to determine how does the self-checklist works? Therefore, I have to observe the learners during learning and ask the students about the self-checklist. I have to research how to develop and conduct observational analysis and interviews.

3 Likes
#5

Clark & Mayer (2011) discussed three methods to research that are used to demonstrate the effectiveness of instruction. As a special educator, I am interested in “what works?”, “how does it work? and “Why it did not work?”

Therefore, experimental comparisons might be the method that I will need to use. I would like to try to work with experimental group and a control group. I could compare test performance of a group of students who learned to use a self-checklist of the steps needed for solving long division versus a group who learn to solve the long division without using the self-checklist.

However, as I read the chapter, I realized a few things: 1) I may not have enough learners in each group, 2) I have to make sure that the treatment and control group are different enough from each other, so I could have an adequate treatment implementation, and most importantly, 3) I need to make sure I plan for confounding variables.

In addition, I want to determine how does the self-checklist works? Therefore, I have to observe the learners during learning and ask the students about the self-checklist. I have to research how to develop and conduct observational analysis and interviews.

#6

Going through the IDD program we are taught to analyze different instructional methods and tasks and how to provide the best training solution. So like @sjmasline (and I assume many others) I align myself with number 3 – How does it work.

Budgets play a large role in many instructional programs. In my professional experiences, budget cuts have plagued our training programs, limiting travel and funding for much needed training. So a greater focus on quality training that is effective is a must. Limited time to train requires us to produce training that meets the objective that is, in many cases, in a compressed time frame. To maximize this opportunity to train we have to understand the effectiveness of a learning process.

In producing new or reviewing current training programs it would be valid to research successful programs, especially if they are in a similar field. This could give you valid information on the effectiveness of a training program and also when it works. I would hope at this point you would have already gained the insight if your instructional methods cause learning.

I agree with Clark and Mayer (2011) that observation is important in discovering how learning best works. I also think utilizing level 1 and possibly level 3 survey results from both the learning and supervisor could help provide information on if the learning program was effective and efficient.

3 Likes
#7

@nrhudson It is definitely a sad fact that budgetary constraints plague most businesses that need to expand their instructional programs. That is why it is important to have a great team that can effectively put into place a quality program for each area that needs to be addressed in the workplace, from basic beginning employment information to continuing education practices. I too agree with you that ‘how does it work’ plays the most important role in relaying the information to the employees.

4 Likes
#8

Instructional effectiveness is important for everyone in the work force. After reading through the chapter, I too feel that ‘how does it work’ plays an essential part in the learning process.

Working in the medical field, there is usually ongoing research in a number of difference facets within each department. Looking into how a particular medication’s effectiveness, for example, plays a large role in research in the hospital setting. This is especially important when it comes to the patient’s length of stay. A lot of budgets are made based on insurance reimbursement for a particular diagnosis code. When the patient’s stay extends beyond what the insurance will pay, every department begins looking at ways to shorten the length of stay.

Most of the time, observational analysis of the particular research program comes to the forefront of how the research is being conducted. Many research studies have several questions that are being recorded for each patient, from their vital signs to length of time on mechanical ventilation for example. Staff surveys also play a role because a lot of times the research company will give a survey to the staff for their opinion on how they think the research is being conducted and how it is effective to patient care.

#9

@nrhudson For new IDDers, you need to say what Level 1 and Level 3 is. Kirkpatrick designed an evaluation process that is used in training. There are four levels: 1. reaction; 2. Learning; 3. Transfer; and 4 Results. [Here is a PDF if you would like to read more] (http://www.4cleanair.org/Kirkpatrick.pdf).

2 Likes
#10

Last term in 660 (Evaluation), we studied a type of evaluation that was criticized for involving stakeholders in the evaluation process. Do the staff members have the necessary knowledge to determine if the research is being conducted effectively? Shouldn’t they be using a professional?

1 Like
#11

Thanks @sjmasline for pointing that out. I was just assuming we all knew Kirkpatrick levels. Think my assumption was wrong.

1 Like
#12

I am also crazy studying girl, so I am trying to remember and throw as much IDD lingo into everything that I can.

#13

@sjmasline Usually before the research process begins the company will send representatives to inform the staff of how the research will be conducted and the hopeful outcomes. There is constant contact between the company and the medical staff performing the study throughout the entire process.

#14

####Which research question (p. 52) do you think that you might be most interested in studying and why?

  • As the authors have shown us and many classmates agree, using evidenced-based methods is the key to instructional effectiveness. Experimental comparisons using controlled variables should always form the cornerstone of research. Because I am interested in education for nurses, nursing students, and mental health patients, asking when does it work is most appropriate.

  • For example, I may need to create a psychopharmacology presentation for experienced psychiatric nurses and behavioral health technicians (BHTs). The nurses start with a high level of knowledge of this area, while many of the BHTs are starting from scratch. I would want to limit my research to studies that compare instructional methods to those with or without background knowledge. Therefore, using factorial experimental materials or environmental comparisons peak my interest.

####What tools or methods might you use?

  • Of course, I would start with the USA Biomedical Library database. Using the keywords instructional effectiveness, meta-analysis, and nursing would provide high quality research on effective techniques for this population. Replacing nursing with behavioral health technician, I would then look for experiments on educational tools for this population. I would do additional searches to see what works in teaching the subject of psychopharmacology.

  • What I’ve initially found is that problem-based learning (case scenarios, etc.) are effective with both populations (Cook et al, 2013). Using [concept maps] (https://en.wikipedia.org/wiki/Concept_map) may also help the nurses (Nesbit & Adesope, 2006), but simulations may be more successful (Shin & Kim, 2013). Also, I need to remember the variation in educational levels of BHTs in the same workplace. As I continue my research, I’m sure I’ll gain a better understanding of what works.

[](https://s-media-cache-ak0.pinimg.com/564x/85/23/06/852306bb111a3fd277c3b8a05f3cb43b.jpg)

####

References

Cook, D. A., Hamstra, S. J., Brydges, R., Zendejas, B., Szostek, J. H., Wang, A. T., & … Hatala, R. (2013). Comparative effectiveness of instructional design features in simulation-based education: Systematic review and meta-analysis. Medical Teacher, 35(1), e844-e875.

Nesbit, J. C., & Adesope, O. O. (2006). Learning with concept and knowledge maps: A meta-analysis. Review of Educational Research, 76(3), 413.

Shin, I., & Kim, J. (2013). The effect of problem-based learning in nursing education: A Meta-analysis. Advances in Health Sciences Education, 18(5), 1103-1120. DOI:10.1007/s10459-012-9436-2

4 Likes
#15

Thanks for pointing that out. :slightly_smiling: I’m a nurse practitioner student and this is my first IDD course. Instructional Design sounds like an amazing field; I never knew of these resources!

#16

I’ve learned this lesson as I transferred from Intensive Care Unit (ICU) nursing to the mental health field. Chart reviewers have no problem informing a treatment team that a patient’s “number of days” (coverage from an insurance company) is over. This factors into treatment decisions on a regular basis!

1 Like
#17

While reading this chapter and in contemplating the research question that most interests me as well as tools I might use, I kept thinking about evidenced based practice. Therapists (I’m sure educational professionals as well) must implement treatment protocols based on evidenced based research with measurable outcomes. So, in analyzing the research questions I was thinking along those lines. I readily identified with all of the questions and the validity of them. Asking what works is valid for identifying a useful tool, exploring when it works is helpful for implementing the tool appropriately, and identifying how a tool works is necessary for executing the tool effectively as well as possibly providing an explanation.

I predominantly prefer the "what works?” research question using the experimental comparison as I agree with education research specialists that it is a powerful method as stated in Clark & Mayer (2011). There are lots of variables to contend with many factors to identify and try control along with finding enough participants who meet the appropriate criteria, all of which can make this a difficult method.

1 Like
#18

@sjmasline I absolutely agree! I also agree with your points on the drawbacks of an experimental study.

#19

I like your examples of problem-based learning and the graphic about concept maps.

1 Like
#20

I definitely agree all of these questions are valid. I think the question you need target will be based on what is already known. For example, you know “what works” but not how it works.

1 Like