BAMI: Encoders

Created: 2021-01-14T18:16:31-06:00

Return to the Index

This card pertains to a resource available on the internet.

Deterministic: given the same input conditions you get the same outputs.

Cochlear hairs are basically audio filters. They receive a certain band of sound and activate cortical neurons as a response.

Semantically similar SDRs should have similar encodings. For example the number 23 and 24 should have a high overlap.

Encoders should be deterministic.

Encoders must have the same characteristics for all values. They must all be the same total size and the same total number of hot bits.

Encoders should have a minimum of 20-25 hot bits per output.

The size and number of hot bits in an encoder is usage dependent. Experimentation or optimization algorithms are needed to find the best values.

Complex encoders made of many smaller ones are created by just concatenating the SDRs together.

Webber's "Word SDRs" for using HTMs on language

Simple encoding of numbers

Inspired by the layout of the cochlea.

Numbers have a degree of overlapping bits.

As a bit array this appears as a snake of 1's in a sea of zeroes.

You must select a minimum and maximum number when encoding in this style.

Example encodings:

Random encoding of numbers

For a given input number use a hash function to determine which bit belongs to this specific value. Create an SDR with this bit hot.

OR this SDR with as many previous numbers as necessary to make sure the required number of hot bits are set.

What this does is ensure that for each digit change in the input number a single bit is moved in the SDR.

Log encoder

Take the logarithm of a number before feeding it to one of the other numeric encoders.

Supposed to make differences in large numbers have smaller effects on the generated SDR.

Delta encoder

Take the difference of two values before feeding it to one of the other numeric encoders.

Supposed to help make predictions on the change of values instead of their absolutes. ex. for values which are constantly changing.

Categorical encoders

Ordinal

Each category occupies the same number of hot bits but no bits overlap between choices.

The total number of bits are the number of hot bits times the number of categories.

Cyclical encoding

Allow hot bits to wrap around; as the value approaches maximum some bits flow over to the beginning of the SDR.

Allows for ex. saturday to turn in to sunday on a week chart.

Spatial encoding

Given a particular coordinate in space: select a region of neighboring tiles until you have enough tiles to satisfy the amount of hot bits which must be active for the SDR.

Run each tile coordinate through a hash function to determine which bit represents it in the output SDR.

Sidenote: I'm not strictly sure you need to use the hash but the book tells you to.

Possibly subsampling these so not all of the extra coordinates trigger bits in the output.

Spatial encoding w/ speed

Increasing the radius extra tiles are selected around the starting point based on how fast an entity is moving.

Subsampling from this larger radius until you have cut down number of bits to the maximum hot bits allowed in the SDR.