Your Guide to Evaluating Security CBT Programs (Part 1)

Note: At the end of this series, I’ll provide a free tool that you can use to make your own evaluations easier, so keep an eye out for it!

With the variety of Computer Based Security Training (CBTs) out there, it can be tough to make a decision around what to consider when choosing the best CBTs for your Enterprise. We often get questions on how to best compare Vendors on Enterprise Security CBTs such as, “What criteria am I supposed to look for?” or “What’s a good way to differentiate?”. To help the community out, I’d like to share a short blog series around what we feel is important when choosing a training program and the advice we’ve given our current and potential clients.

In Part 1 of this series, we’ll discuss what you might consider when it comes to evaluating course content for your students. Feel free to borrow these ideas for your own evaluation criteria and tailor them to your needs!

1. Does the Course Content match your Team’s Goals?

More content does not always equal better value.

  • High Score — Content covers your educational goals
  • Low Score — Does not cover required concepts

One common way I’ve heard people evaluate course content is comparing between vendor content, then picking the one that offers the most amount of content. This seems like a great idea, more content seems like better value but we now know that this is not true. In our experience, too much content does not mean great learning. In fact, it often leads to the complete opposite.

To be up front, way back in 2008 we started our CBT courses with a lot of content and the feedback we got over and over again was that the courses were just too long. After taking the course for a bit, students just stopped wanting to learn. Instead they went the button mashing “next, next, next” route until the course was complete and all our intended learning goals were overlooked simply because the student felt the the CBT was… heavy. We discovered that this can’t be how you truly inspire learning and our courses have now changed for the better.

One effective approach is to first identify what key learning goals are important to the security of your organization. What are you trying to teach your teams and what is important to students? For example, do you have a specific concern around SQL Injection and Cross-site Scripting? If so, then make sure those items and their learning rank higher on your course content evaluation. Ensure that students will at minimum understand those concepts after completing this Vendor training.

Many CBT programs will try to be a jack of all trades, but try not to make this the goal. Your team’s learning should be targeted to your end goals of securing your organization and ensuring that students use the training to achieve their goals.

What to look for:

  • Modular content — Ideally modular in 15 minutes chunks or less to allow busy students to absorb content when they have time. (like a productive coffee break!)
  • Targeted Content — Course content should meet your student’s learning goals and shouldn’t stray too far from what they are meant to learn. Students should come away clearly understanding what they had set out to learn.
  • Up-to-date Content — Content should be up to date and relevant to today’s standards. This can be hard to evaluate so it can help to get someone technical to review material.

What to avoid:

  • Heavy on content — too much encyclopedic content is troublesome and will not benefit your students. Today’s students just don’t have time for this.
  • Mismatched Content — generalist content should not appear in specialist courses and the opposite is true too. For example, developers need to learn about secure coding practices but heavy developer code should not appear in an awareness course.
  • Unclear Audience — within the first 10 minutes of the course, it should be clear who the audience is. A developer should not feel patronized and general staff should not feel as if content is over their heads.

I hope that this helps you establish some basic criteria to evaluate a Security CBT Vendors on Course Content. Remember that a priority should be to first determine what you want your teams to learn. Taking a little time to research will make evaluating any vendor’s material much easier.

Stay tuned for more as part of this blog series!

Previous Article
How simple reviews can increase your Security Education results
How simple reviews can increase your Security Education results

We all think of deploying amazing Security CBT to our teams, envision them taking it, loving it and your co...

Next Article
Debunking myths: Security Awareness is Useless
Debunking myths: Security Awareness is Useless

Last month, a story ran on Dark Reading around why security awareness is useless. I cringe reading such sto...