Dependency Graph For Knowledge

Can the knowledge or skills in a field be broken down into discrete atoms of knowledge?

If it can, can we:

  1. Teach those atoms of knowledge (or skills)
  2. Assess people's current abilities in those atoms
  3. Assign some dependencies between them
  4. Help students to retain (and strengthen) the knowledge/skill over time.
  5. Define and measure "motivation".

And, once there are participants, can we use statistics to:

  1. Find which teaching methods are more effective
  2. Find which assessment methods are more efficient
  3. Improve the dependency graph itself
  4. Measure the effectiveness of retention techniques
  5. Predict and optimize "motivation" over time.

And:

  • Given a goal, suggest (and monitor) a path to achieve it
  • When an atom of skill/knowledge is conquered, suggest one or more atoms to do next.

What is a knowledge dependency?

At the most gross level, I've seen this kind of dependency:

The subject CS101 is a pre-requisite for CS201. In order to sign up for CS201 you must have accreditation in CS101 or an equivalent subject.

But that's far from "atomic".

Atomic would be much smaller. And the dependencies are not necessarily "hard" dependencies.

I don't think it's always true to say "You can't learn Y until you've learned X". Because our human brains are pretty clever and can learn a lot even when given imperfect information. The brain can fill in the gaps.

It might be better to say:

It is helpful to learn X before you learn Y

or

Learning X before Y will give you a better chance of learning Y

...where "better chance" can be defined in a way that is measurable. For example better might mean:

  • People are likely to learn Y more quickly
  • People are more likely to complete assessments of Y without quitting
  • People are more likely to achieve a mark of P% on assessments of Y, within a timeframe of T.

And "the system" could "automatically" learn these kind of things about people.

Whether they are true about a specific individual would remain to be seen. But a level of confidence could also be calculated "automatically".

The Principle of Atoms

Atoms are from an ancient greek word meaning "un-cuttable".

They are the smallest imaginable unit. (Never mind that the physics of the real world out-stripped the imagination of physicists. The things they called atomic did turn out to be cuttable, in the right circumstances.)

The idea behind looking for "atoms" of knowledge is to divide and divide knowledge until it can't be divided any more. The very small pieces of knowledge that remain are useful because they can be dealt with in a clean way. They can be assessed quickly, taught quickly, rearranged easily, and statistics about them can be gathered easily.

Contrast with what happens when knowledge is grouped into larger modules. You can't assess the module until the entire module is complete, so cycle time is longer. (Shorter cycle time leads to a system which can be improved dramatically faster, see)

If a module is not working (according to the stats) you can't tell which part of the module is the problem. You can only replace the module in its entirety (which is expensive) or tweak part of the module, and wait to see if they move the stats in the right direction.

Another common size for the units of learning is the "lesson", based on a class room or lecture hall lesson, and taking 30 to 90 minutes. This is also big enough to slow down the assessment. Instead a lesson should be thought of as a "molecule" made of many atoms put together in a specific way. Constructing the right atoms, and putting them together in the right way is a big challenge.

A side-note on arrows

Skill Y depends on skill X. Do we write:

Y → X

or:

X → Y

The business of arrows is a tricky one.

What is it that "moves" in the direction of the arrow? Knowledge which moves from X to Y, or questions which Y sends to X.

I think it's more intuitive to say that:

X → Y

Skill X "leads to" Skill Y.

Or, if the system can have more than one type of arrow (or information about arrows) then labels can be attached to arrows:

X –(leads to)→ Y

If needed, there are many other arrows available in html:

⇝ ↠ ↣ ↬ ↷ ⇁ ⇉ ⇏ ⇒ ⇛ ⇢ ⇴ ⇾

...which could each be put to special uses.

It has to be open and editable

For such a knowledge graph to be valuable it would need to be open source, probably licensed under a Creative Commons License, and be willing to accept pull requests from others (or permit them to fork the graph).

Retaining knowledge over time

A level of skill maintenance is required if you want to retain your skills.

Systems such as spaced repetition try to do this in an efficient manner.

There is a principle here:

knowledge (or skills) are far more valuable if they are retained.

It could be stated in a more extreme manner:

There is no point remembering something for a test and forgetting it afterward. Any system that encourages that behavior is worse than no system at all, as it wastes effort that could've been used for a beneficial purpose.

If spaced repetition is used, and knowledge is atomic, you might think to ask: is an atom of knowledge the same as a single card (or fact) in a spaced repetition system. Sometimes yes... sometimes no.

Sometimes a single piece of knowledge should be broken up into several cards. They may retell the same knowledge in different ways. It's really a technical decision related to how people store knowledge, and the best way to use spaced repetition software. For now just think of it like this: the model used should allow an atom of knowledge to be linked to by more than 1 spaced repetition fact.

Examples for 3 cards required for a new term:

  • Word ↷ Definition.
  • Definition ↷ Word
  • "Cloze deletion" example. (may have several of these).

Meta example:

(Let me use the nomenclature, "Front of card" ↷ "Back of card")

Card 1 (Definition)

  • "Cloze Deletion" ↷ "A fill in the blank style of asking questions"

Card 2 (Definition)

  • "A fill in the blank style of asking questions" ↷ "Cloze Deletion"

Card 3 (Example)

  • "Questions with missing words are examples of ____ deletion" ↷ "Questions with missing words are examples of cloze deletion."

Even though we could have many cards on that one piece of knowledge, the knowledge fragment is what I would consider atomic. (But metrics could reveal if it is learned and retained faster if always agglomerated with other knowledge)

Metacademy

The closest thing I've found to the knowledge graph part of this idea is "Metacademy" which looks really interesting.

It has a knowledge graph, it is open source and CC licensed. (Great, great and great).

I wouldn't quite say the pieces of knowledge are atomic. They are more like a "lesson."

There's a lot of fundamental knowledge assumed.

As a result the knowledge is many sets of disjoint networks rather than a single mighty network. This is just a literal network effect though (as more detail is added, the number of nodes would grow but so will the number of edges, and the connectedness of subnets)

The bar for adding a piece of knowledge to the graph is quite high. I think I'd want that bar to be quite low. (Particularly for the first few thousand nodes)

Knowledge Graph example from Metacademy

I don't think there's any assessment built in, nor (consequently) any spaced-repetition.

Each item has:

  • Summary
  • Learning goals
  • Prerequisites
  • Resources
  • See also

Resources can be referenced by multiple items and have:

  • Title
  • Authors
  • Brief description
  • Resource Url
  • Resource Type
  • Access (Free|Free with registration|Paid)
  • Resource year
  • Edition Years
  • Extra notes

Motivation: Measure and Optimize

Fundamentally: How is motivation measured? How is it optimized?

Motivation - maintaining this should be inherent in the system and the metrics that drive it internally

No point having a perfect system if it’s too boring to use.

  • Have to ask why they want to do it - why do the participants want to engage with it?
  • Why is it engaging? what do each of the students consider fun on that day at that time?
  • Do personality types exist (Ray Dalio thinks they do, I'm less convinced) but if they do exist, how should they influence a dynamical system's delivery of material?

Gamification

Principles that have become mainstays in gamification could be successfully employed in the system I'm imagining.

Knowledge.... or Skills?

I've used knowledge and skills fairly interchangeably throughout. What do I mean by this? And should it really be one or the other?

I think of them as somewhat interchangeable because:

I give a specific meaning to "knowledge" which separates it from "information" or "data" or "facts" and elevates it above them.

By knowledge I mean "applicable" or "useful" information. Information that has been sufficiently engrained into your mesh of understanding and behavior that you will apply it at the appropriate time.

Here's an example.

Imagine an alien has arrived in human form on the earth. Her name is Trevor, though that it is irrelevant*. Let's imagine that Trevor has previously learned the fact that sunlight is warm. She steps outside on a summer day and is surprised by the warmth of the sun. In such a case Trevor only had information or facts: she didn't have "knowledge". Knowledge is more like a skill: it is so internalized, so embedded within your world view that it is immediately recalled at the appropriate time. It may even become part of the "unknown knowns" -- the things you have internalized to such an extent that you don't need to consciously consider them.

* Even though her name, Trevor, is irrelevant, it is still true. Hi Trevor.

References

See also