Needs Assessment

Your Training Department needs a Needs Assessment
Over the past several years I’ve been working at several high-profile technical start-ups. I joined those companies either as an individual contributor or as a contractor – rather than in a Management role. I enjoy the challenge of creating and teaching technical curriculum. But after my experiences at these companies, and seeing their struggles, I’m thinking about getting back into a Training Management role.

In each case, the companies seemed to be flailing in their efforts to create a well-functioning Training department. I would occasionally offer advice, but, for whatever reason, the existing teams seemed incapable of listening. I saw Training VPs come and go, Training Directors come and go, Training Managers come and go – but they would only leave after they had run out of individual contributors to blame and fire for the problems which arose. These were problems caused by a lack of good Leadership. And that has prompted this post. I want to describe a way to think about organizing Training. I spent a couple of years as a Product Manager. I also spent a couple of years in Technical Marketing. The result is that concepts like “distribution channels”, “market segmentation”, and “positioning” are not just buzzwords to me – and they can reflect deep thinking about Strategy. So understanding these concepts can help a Training Department, since they can help define both the tactical and strategic goals of a Training organization.

The Typical Reactive Training Approach
When I look at all of the start-ups where I’ve been working, they hired somebody with “Training Director” or “Training Manager” experience on their resume, and then hoped for the best. In every case, their logic seemed to be: “We know that we need Training and we know that we need eLearning since we get requests from customers.” The reaction to these customer requests was hiring or contracting some Managers, Curriculum Developers, Instructors, and eLearning developers. Then, based on whatever customer requests had come in, or whatever Senior Management thought Training should be doing, a bunch of projects would be launched.

Well, that is both very expensive and very wrong. That isn’t planning – that is a reaction. And, even if successful, is it the success you really want?

Trivial Market Segmentation
Let’s instead look at this from the perspective of Market Segments. Let’s take a simple classification of customers – categorized by potential for future sales.

  • Tier 1 – “big companies”
  • Tier 2 – “mid-sized companies”
  • Tier 3 – “small companies and students”

When you think about company goals, the focus of a startup is often short-term profit and seeding the mid-term customers. And, candidly, ignoring companies that might become a big customer in five years – since focusing too much energy on them right now might mean that you won’t be around in five years. On the other hand, there may be other goals. That is why it is important to have ways to think about these things. So, from a Training perspective, how might a company meet the needs of their customers? Here is one way you might structure your Training:

  • Tier 1 – Instructor Led Training from your in-house instructors followed by Consulting engagements
  • Tier 2 – eLearning for purchase and Instructor Led Training (maybe using contract instructors)
  • Tier 3 – Documentation and perhaps free eLearning – and perhaps meetups and video on your web site.

So, now you can fill-in this chart with your own approaches to meeting the needs of each market segment. It might include self-study and video for a Tier…or be exclusively documentation for every Tier. You should fill-in this form so that you are aligned with the strategic goals of the company.

For most, it would be ideal if you could enable everyone to be successful at deploying your product (assuming that is your goal.) But, in general, trade-offs must be made. So, after figuring out where you are, then you can figure out where you want to be – and what form of Training you ultimately want for each Tier. (I sometimes use a “horizontal” and “vertical” chart of skills to help focus development efforts. Again, something I picked up from my time doing Technical Marketing.)

The Mistakes of High-Flying Start-ups
The most interesting situation was at one company that I left. They did this:

  • Tier 1 – Instructor Led Training mostly by contractors
  • Tier 2 – Instructor Led Training mostly by contractors
  • Tier 3 – Instructor Led Training mostly by contractors (with some free videos to promote the Training)

They are really only serving Tier 1 – since rich customers are really the only ones that can afford the expensive Training. They consider Training a success since they are generating lots of revenue from it. Their focus on revenue has also meant that they hired Curriculum Developers who don’t have the Training chops to create training that meets serious objectives. The result is that, while the classes get good reviews from students, they don’t really teach all the skills needed to successfully deploy the product. So the company is making lots of money (largely from Training) and they have few complaints from customers. As a result, they “know” that they are doing Training right.

Seeing what was going on, I had a casual chat about company goals with the CEO. His perspective was that we were there for the investors, and those investors wanted to cash out with a good return. Essentially,  I left the meeting understanding that maximizing the return on Training is his primary focus.  Ironically, they could have achieved that and much more with just some slightly better hiring decisions and project planning decisions.

Another company I joined did this:

  • Tier 1 – Instructor Led Training by contractors
  • Tier 2 – Instructor Led Training by contractors
  • Tier 3 – Instructor Led Training by contractors (with some free videos to promote Sales)

Notice a pattern here? This is the same thing the other company did, but with a few new mistakes added in. This company had a Training Manager that didn’t understand that instructors must be properly ramped-up, and didn’t understand that course material must at least give students the impression that they are learning enough. The result? Lots of refunds, cancelled classes, and fired staff. I should add that this company allocated time from their best engineers, and the CEO allocated lots of company resources to create the training “fast” – but that just doesn’t help when you don’t know what you are doing.

But, even if their course had been successful, is this what the company should want? Is this type of training the most effective for the short-term goals or the long-term goals of the company? No. Not even close. Instead, they think that Training is just creating some courses and eLearning. And the Venture Capitalist CEO thinks that creating it faster is being “customer needs focused” – and thinks that he is being a more effective CEO by personally insisting on fast development. But with the time, money, and energy they have now spent creating and re-creating useless course material, they could have created a world-class Training organization. Instead, they are still floundering as I write this.

Another company I recently left did this:

  • Tier 1 – Free eLearning
  • Tier 2 – Free eLearning
  • Tier 3 – Free eLearning

This is interesting in that it meets one important goal of the company. They want a competitive advantage in sales situations. And, as always, free is a good price. So, in some ways, this is genuinely successful. But they decided on this strategy after trying ILT and having problems in class after class (due to hiring the wrong people). So this is really their fallback strategy. But, obviously, this is a flawed long-term strategy. Their customers get no advanced training and get no seminars with live instructors who can help their development projects succeed after the introductory training.

So another case where they lack a cohesive Training strategy and lack the right staff to implement a long-term strategy.

Best: Alignment with Tactical and Strategic Company Goals
In the end, I am just arguing for something obvious. The goals of Training should be strategically and tactically aligned with the goals of the company. In addition, projects should be prioritized in a way that recognizes the trade-offs of that prioritization. Of course, all this is predicated on hiring the right people to craft and implement that strategy. (You might check out my “The Most Important Training Thing I learned in College” post.)

I’m arguing that a “needs assessment” should be done for Training Departments and companies – and not just students.


Meetups are not Courses


, ,

I was recently attending a technical Meetup and I noticed that the presenter’s slides made a lot of the same errors that I see in some technical courses.

Lack of Practice Time
I noticed that the Meetup presentation had a list of information about the product at the beginning of the presentation. That was immediately followed by another list of product information.

Instructional Design Theory says:
• Short Term Memory is Limited to about 4 items
• People Have to Use Information to really learn it
• People learn from Examples

This means that if you tell people 10 things about your product they will forget half of them before the meeting ends. You can overcome this with spaced practice of the material. Such practice may be case studies (for realistic examples), hands-on exercises, or “quiz” slides with questions to encourage recall of information. Demos can also work. Repetition and spaced retrieval are most effective. (Studies have shown the effectiveness of spaced rehearsal with 10 minute “breaks” between stimuli.)

But you should be quite choosy about what you decide to emphasize. People have limited ability to remember things, and you should only emphasize the important things that you really want them to remember.

Use Information Chunks
The Meetup presented syntax examples by themselves. Sometimes, the syntax would have a trivial example of use.

But theory says:
• People Process Information in Bite-Sized Chunks

This means that you need to group information together into small, meaningful units. So break down slides into groupings that can be recalled as a unit. An example of how syntax solves a problem, for example, is much more meaningful and provides a chuck of information worth remembering.

Focus Attendees on “Need to Know” Information
The presenter treated all slides as equally important — but some basic concepts are really critical, while other details are less important.

Studies have shown:
• Minds wander about 30% of the Time

The typical attention span is under 10 minutes. Before fMRI brain scans, it was believed to be about 20 minutes. But now brain scans show that the mind is wandering before the 10 minute mark. This means that you must let people know which information is important, so that they can focus on the things that they really need. Using “preview” and “review” can help with this — as well as simply telling people what they need to pay attention to.

Build on Existing Knowledge
The Meetup presenter didn’t build on what the attendees already know. Instead, he introduced the architecture of the product without showing it in contrast to the well-known architecture for similar products.

• People Create Mental Models (we create categories)
• People Interact with Conceptual Models

People place new information into categories, and if the topic is completely new then there are no good categories in peoples’ mind, so they have to use their scarce mental energy to create a model. But they are just learning the material, and their model may be wrong. Instead, the slides should make the model explicit. Once the model is created, it is used to “interact” with the new information. It is usually best to find their existing Conceptual Model in order to build on existing knowledge. (And save the mental energy, confusion, and distraction that accompanies trying to build a new mental model.)

Less Is More
Technical presentations routinely have too much information. But new developers must throughly learn the basics in order to succeed at their job. This is important if we want people to be able to apply the knowledge that we are presenting.

It’s true that:
• Memory Uses a Lot of Scarce Mental Resources

In a given time, people have a limited ability to learn. Short term memory isn’t the same as long term memory. That means that most things are forgotten before the end of class. The problem is: some of those things are important. Instead, we need to choose the most important things, and make sure that we communicate and emphasize those things. That way, we can make sure that people are successful in their job. So it’s OK to present a lot of information…but only after making sure that everyone has learned the important skills.

Teaching Scripting


, , , ,

I once worked at a company that had a successful course on their proprietary scripting language. The course had run for years, and I was asked to do a major revision of the course.

Talking to the Instructors

I started out by talking to the instructors who had been teaching the course. I wanted to find out how the existing course had been working.

To my surprise, the instructors told me that every class had several students who weren’t able to code using the scripting language. Apparently, since the language was promoted as “easy to use” and “easy to learn”, several students in each class had no real coding background, and quickly got lost as soon as any real coding started.

From a company perspective, this was a huge problem. If some students were unable to use the scripting language then product deployments were put at risk. And even with successful deployments, some departments would not be able to adapt the product for their use. That had real costs in terms of loss of future sales. A better course can create happy customers who buy more of our product.

Identifying the Problems

I studied the the course material and found three areas where non-programmers would potentially flounder.

• Creating a first program
• Using functions from external libraries
• Doing I/O with formatting

Everything between those “transitions” was really just building on those basic skills. Most instructors observed that students had trouble with various pieces of syntax, but I theorized that those troubles were really based on “cognitive overload” from poor skills at building a simple program. The combination of learning new syntax while also trying to write enclosing code was just too much for students with no coding background.

My Solution

I realized that the students needed more “hand-holding” with the basic skills. Since the typical class was at least 16 students, and lab time was limited, it wasn’t possible for the instructors to have time to help every student during lab.

So I redesigned the slides around the areas of difficulty that I had identified. Basically, I created slides that were designed to be “real time type along” — so students would type in the code on the slide and execute it after the instructor presented it. It took a surprising amount of work to create those slides. I didn’t want any slide to be more than eight lines of code (I got most down to 5 lines), and that took some careful selection of syntax in order to minimize typing.

Immediate Practice Works

One big benefit was that this provided immediate practice on the presented syntax. And practice is associated with a movement of information from short-term memory to long-term memory. Short-term memory was “freed” for new information. So it became possible to present more material. Even though the class spent more time typing as a group, more information could be presented. It still seems ironic to me that slowing down the material actually allowed more material to be presented.

Real Time Feedback During Delivery

Another benefit is that the instructor would provide immediate feedback to anyone who was having trouble with any of the short programs (there were a few problems in almost every class, but as you helped one attendee you typically solved the problem for everyone having similar problems.) Honestly, this had been one of my big concerns as I revised the course — would the instructor be overwhelmed helping students? As it turned out, it became a more efficient way to help students. And with 5 line programs, the solutions were typically simple.

The Results

In practice, this revision exceeded my expectations. Every student I taught with the new material was able to use the scripting language  — and do far more with it than prior classes had ever done. And it made the class easier for students at the same time – no more struggling to get the basics. Instead, they spent lab time building on knowledge that had already been practiced during class delivery.

Case Studies Add Value


, , , , , , , ,

There are studies showing that software costs much more to maintain than to initially develop. Well, the same is true for Training Courses. They will often go through many revisions over years.

Case Studies Don’t Have to be in the First Release

For most new courses, the first release is really about getting the basic information into the course. It is usually at the first major revision that we can try to add more real value to the course. It is the first major revision where we typically add things like Powerpoint speaker notes. But perhaps my favorite thing to add are Case Studies.

I’ve noticed that very few courses include Case Studies. I think the literature on Case Studies makes them seem very complicated to create, and if you order some case studies from Harvard you will find that they are typically quite involved. But I’ve found that simple case studies can be quite effective — and attendees love them.

How to Create Case Studies

I’ve created case studies in two ways:

1) Go through support logs.  This is very time consuming, and it can take a week to find the basic material for a few case studies, but you know that the situations are realistic if they have happened to multiple customers.

2) Conversations with Customers.  We are lucky in Training — we are customer facing. That means that we have the opportunity to talk to customers and hear about common problems. This is another reason to add case studies after a first, limited release of the class.

The Form of Case Studies

My case studies are typically two slides. A first slide presenting information and a question, and a second slide explaining the resolution. This has the huge advantage of being very quick to create. I don’t try to do anything as complex as Harvard-style case studies.

My real case studies involve information I’m teaching in a technical course, and they would be hard to show here. Instead, here is a somewhat contrived example (I’ve never taught PHP).

Slide One:

Case Study Slide 1

Case Study Slide 1

Slide Two:

Case Study Slide 2

About those Slides

I make a point of presenting extra information on the first slide. In real life, we have too much information that is irrelevant. In this case, the RAM and hard disk sizes are extraneous. It is also routine for relevant information to be missing — in this case, the student has to figure out that the code only worked before on a different operating system.

On the second slide I just present the solution.

How I use those slides

I usually try to have at least two case studies in the morning, and two in the afternoon.  Generally, I present them at the end of the relevant section so that students can recall the information that I just taught. (On rare occasion I’ve used them before the relevant section. I do this when the students seem poorly motivated to learn the material. A good case study can show the relevance of the course material.)

When I present the first slide I just stop talking and wait to see if anyone volunteers a perspective. Since I do customer training, I never call on anyone to answer. I don’t want to embarrass anyone. (And good Case Studies are often quite hard. The discussion of the answer can be a significant part of the course.)

In every class where I’ve added case studies, the student review scores have risen dramatically. People really like to see how the information they learn can be used. And, if I picked good case studies, they may even encounter those situations in their job.

Training Has Changed


, , , , , , , ,

Technical Training was run differently back when I started my career 30 years ago.

Minimize Investment and Maximize Profit

Modern business as driven by investors seems to have one over-arching goal: minimize investment while maximizing profit. For Training, that means that we rarely invest in proper course creation. We contract out most course development and deliver most training through 3rd parties. Of course, anyone who has been around a while knows that this guarantees mediocrity. After all, contract developers work contract to contract, and they are expert on one product for a few months and then expert on another product for a few months. They can’t possibly become expert on your technology. How good can the course be? (I should add that I have seen the system work for some “simple” courses. Typically, courses built around skills that involve mature technology.)

It is even worse on the delivery side. I’ve literally had instructors who deliver Red Hat training one week, VMWare training the following week, Cloudera training after a few days off, and then do a first delivery of our training the next week. How can they possibly become an expert on our product with a ramp-up time of a few days??

Legal Contracts replace Common Sense

Of course, we have contracts saying that we will approve instructors and we have contracts requiring particular instructor ramp-ups, but anyone who works in the industry knows that those contracts are worthless. Each regional 3rd party training manager will do whatever it takes to get an instructor in front of a class — and deal with the refunds and repercussions later. After all, money is on the table, and 3rd parties have the same value system as their investors — minimize investment and maximize profit. (I should add that I have seen this work for some low volume/stable frequency courses. Such courses can get a few instructors permanently assigned. But you really have to get lucky for this to work out well long term. Long term, there is no such thing as a constant frequency of delivery in every region.)

As things go wrong, it is routine to switch 3rd party delivery organizations and fire curriculum developers and fire instructors, telling them that they are incompetent. But it is really the system that creates incompetence in good people.

The Old Days

I remember when I worked at companies that actually screened every potential instructor to my specifications (since I was typically responsible for technical courses.) And I would specify the instructor ramp-up time and ramp-up tasks. We invested in our instructors and supported them directly while they were in the field. Our entire organization was behind them. We did everything we could to make sure the instructor could teach the class well. Sure, things went wrong occasionally, but things went right a lot more often.

It has always been hard to find good instructors. But now the good instructors are treated like disposable assets, rather than someone who will forge a long term relationship with a customer. So even good instructors don’t invest the emotional energy in the company anymore. And it has always been hard to find good curriculum developers. But now the good developers are often treated like factory workers who are expected to crank out material to a schedule — quality be damned. (Well, they are told that quality is important. And they are fired when they aren’t able to produce quality in a system that is designed to produce mediocrity.)

And things are getting worse

As the years go by I meet fewer and fewer people who deliver well or develop well. The skills are being lost because no company invests the time and money to train their delivery staff on delivery skills, or train their curriculum development staff on curriculum development skills. I remember when delivery people were given multiple classes on delivery skills. I haven’t even heard of that for a least a decade.

I sometimes wonder if, at some future date, we will look back at the years spanning my career and describe it as the end of the “golden age” of Training.  Unless companies become something to serve society, rather than being “cash machines” for investors, I don’t see how things will change. It makes me sad. Sad for me, sad for our society, and sad for our children.

Agile for Curriculum Development? No Thanks.


, , , , ,

It seems that there is always another Training Conference to describe how to apply Agile Project Management to Training.

I have a software development background, so colleagues sometimes ask me if I’ve ever worked with Agile. Since Agile is a collection of techniques based on best practices in software development, I’ve used Agile techniques such as card walls, feature lists, feature prioritization/time estimates, nightly builds, short build cycles followed by demos, and probably many others that I’ve forgotten.

How is Agile used in Training?

Whenever I speak to Training people who use Agile I keep hearing one technique over and over again — daily stand-up meetings. When I ask about other Agile techniques that they are applying, they usually don’t use any of the others. Use cases, user scenarios, and schedules with deliverables I don’t really consider to be Agile specific — they have been part of Curriculum Development for longer than I’ve been alive. The only thing that usually seems to be added in the name of Agile is daily stand-ups.

Why should you have daily stand-ups?

I think of Agile as a set of techniques that a development group can “mix and match” to improve a project — not something that managers impose on their developers. The common reasons given to have daily stand-up meetings are:

• Present any obstacles to your progress
• Receive and give suggestions to overcome obstacles
• State what you accomplished yesterday
• State what you are planning to accomplish today
• Generally help each other as a team

Looking at this, it is obvious that these will vary in importance with different types of projects and different types of team members. Why would anyone think that a single approach would be the right approach for every type of project?

So what are the characteristics of Curriculum Development?

A single technical course is typically put together by an individual. That is, each Curriculum Developer tends to specialize in a particular technical topic while developing a particular course — it is rare that any other team member has any depth of knowledge on the same topic (if he did, he would probably have been assigned the course.)

Course obstacles tend to fall into some fairly standard categories.  As a senior Training Manager, I can’t remember the last time that I didn’t predict a potential problem area and already have a fallback plan waiting for that possibility.

Courses, in order to be created successfully, must start with a loosely defined period of exploration. During that period, typically 2 or 3 weeks, the topic must be investigated in order to identify what will be covered and understand the difficulty. That means that after the exploration period an initial project plan can accurately predict what happens in each week of ongoing development.

And what does this mean for stand-ups?

If experienced management allows the accurate prediction of potential obstacles and the creation of a plan for response to those obstacles, there really isn’t much reason to force people into a daily meeting to figure those out. And if you have professional employees, they will let you know when something has gone wrong and which fallback scenario is being used.

If a well thought-out plan can accurately predict the flow of work, there isn’t much reason to have a daily meeting to tell people what they are working on. Just look at the plan.

Helping each other as a team is a good thing.  And I do like to have people come into the office at some frequency in order to foster interaction. But it is hardly worth a daily meeting.

(Having said all that, there have been rare occasions when I’ve scheduled daily meetings and required extensive status reporting. It depends on the specific project and the specific staff. There is no “one size fits all” solution to real-world problems.)

Managing Training Projects



I’m a big believer in personal autonomy and responsibility. And that leads to some difficult situations.

Planning Projects

I’ve gotten pretty good at project planning over the years. So at a first meeting to define a project with a direct report I typically arrive with a “straw man” plan.  I have a fairly standard form that I’ve been using for a number of years, and it has generally been pretty accurate at helping to define project boundaries and anticipate potential problems.

I know that many people have trouble with planning, so I try to offer some helpful guidance. But I’ve always allowed my direct reports to ignore my straw man plan, define their own project plans, skip most milestones, avoid status reporting, ignore any input or advice I may give them about how to be successful in their role, and generally do whatever they want with their time.

I should add that this type of planning is different from mentoring new Curriculum Developers, new Instructors, or new Programmers.  For people new to their job role, I use completely different techniques of feedforward and feedback to get them up to speed.

The Minimal Plan

For most projects, about the only thing I absolutely require is the definition of some deliverable in some timeframe. Usually, for any non-trivial project, a couple of weeks of work are required to establish the deliverable and the timeframe. Those weeks are usually spent investigating the “dark corners” of the project. I should add that that we are not talking about pure R&D projects. Basically, a modified Waterfall works pretty well in most environments. While there are always “undefined” components, they can typically become defined over a fairly well defined period.

Now, I do have some advantages here. I’ve personally created more than a dozen courses, written code in some major development projects, and done program management within and between companies, so I generally know when I’ve allowed a generous schedule.

But Cracks Can Appear

The problem typically first arises when I’ve casually asked “How is it going” and I get an answer like “I don’t think I’m going to be done on time.” That is usually followed by a flurry of excuses.

At this point, I ask him to bring in all his work so we can go over it and try to figure out the problem. Sadly, in each of these cases it quickly became apparent that he was treating our professional work environment as a vacation.

Solving the Problem

The solution is simple – even though it is rather unpleasant. I arrange a 1 on 1 meeting to communicate:

• We have a department that treats everyone as professionals and allows them the freedom to schedule their own project and keep their own hours.
• This environment can only work if we treat our commitments seriously.
• If their commitment to our department doesn’t change quickly, I will fire them.

The Results

So, how has this worked in real life? I have only had to have meetings like this twice. In both cases, the employees went on to become model employees. And in both cases, the employees agreed that they had mistaken our “high autonomy professional environment” for a “low accountability” job.

How I Marketed My Courses


, , , ,

For a few years I ran my own one-person Oracle Training Company. By that point in my career, I had managed a couple of small Training groups and even been a Training Manager in a Fortune 50 company — so I already knew the Training business pretty well.  I’d also done a 2 year stint in Technical Marketing — so I had created ads, positioning papers, competitive analysis, and technical white papers.  I was ready to strike out on my own.

My Product

The first thing I had to do was figure out my product differentiators, and how to position those differentiators in the minds of my potential customers. As it turned out, it was pretty easy.

My Differentiators

• High Quality Training
• High Quality Course Material
• High Quality Instruction
• Information and perspective not available anywhere else

Now, this might sound like the marketing claims of just about every training organization on the planet. But I had worked at “world-class” Training organizations, so I knew that these benefits existed almost exclusively in the minds of Marketing, Sales, and Training Management.

So I was really selling myself, my skill set, and my ability to deliver on those promises.

Positioning – My Strategic Advantage

On one hand, it was easy.  I had worked as a Senior Engineer on some pretty complex projects — and I had worked on the Oracle Source tree. I had instrumented the source code to characterize SQL performance through various code paths. How many Oracle Trainers have done that?

And I have a great Training background. I had been mentored by a team of Senior Navy trainers who were running a small training group. So they had drilled into me the techniques to create well structured material — material that was designed to be effective for students in their jobs. Most Curriculum Developers that I’ve met in industry are self taught, and their curriculum is usually just a “dump” of technical information with a few diagrams. So my background in Curriculum Design was first class. How many Oracle Trainers can say that?

And when I worked at Oracle I had also created training for the Oracle Consultants. I taught performance tuning and a few other “obscure” topics so that they could help Oracle customers. How many Oracle Trainers can say that?

So I had all the advantages. I knew that I could create better training with better delivery and, unlike most large Training organizations, I actually cared about my customers.

Selling my Courses

I’m not a smooth sales type. But I had created copy for ads, so I did a competitive analysis to round out my positioning, and I created a brochure.  That was perhaps the hardest part of launching my Training company — I spent months on my three brochures (three since I wanted to test them for effectiveness against each other.) Then I bought some lists from a direct mail broker, and $20,000 dollars later my business was off and running.  I had more business than I could handle.