| ™ |
Neural Network Studies
by Tom Veatch
Neural Networks are an important idea of our time:
- An introduction to neural networks should enable any motivated
high school graduate to understand them completely and correctly,
as well as develop appropriate intuitions about what is going on
inside them as it processes inputs and as it gets trained.
I offer you this accessible mathematical introduction
to NNs.
- Why don't folks use "Newton-Raphson", better named Successive Linear Approximation as the update
rule for NN training? Instead, they say, crawl off in the
right direction: does crawling make sense to you when you could
jump most of the way there in one step? With an algorithm we charitably call Newton's method, we should
rather make a good-to-excellent guess how far to go (namely,
\(1/e''\) times the slope \(e'\)). In case this is because folks
think the math might be hard or unknown for Neural Networks, I
offer you the derivation, along with
arguments for its likely superiority. Now there's no excuse not
to try it.
- Fuzzy logic is logic where truth isn't just 0 or 1 but any number
in the range of [0..1]. In Neural Networks
as Fuzzy Logic Engines, it is pointed out that neural network
nodes can be interpreted as Fuzzy Logic Predicates (with outputs
in the range [0..1]). Then the idea is contemplated and advanced
that they can be re-engineered to learn Fuzzy Logic relationships
so as to actually be, learning fuzzy logic engines. Neural Nets
are enhanced by the semantic interpretability and the logical
operators of Fuzzy Logic; while Fuzzy Logic benefits by the
data-driven learnability of Neural Networks. FLENNs!
- Thinking about spatial perception as sound, or rather as echoic,
two-signal sound, the ideas came to me of Space Representing Neural Networks. Inspired
by my experience, once upon a time, being electrocuted (no not to death), and
"seeing" the discharge path inside my body, this is one explanation
of how an organism could have such an experience. An SRNN is a
multi-dimensional, multi-scale, bi-directional, space/time
representation and transformation system which enables capture
and combination of information regarding position, size, shape,
movement in merged perception from multiple sensors as well as
the reverse, namely, self-assigning, choreographed activation
trajectories for actuator-driven movement within space as a
representational target for action planning, learning, and
control.
- Three ways to implement the geometric transformations needed in
SRNNs are offered in Dual Quaternions
(namely Quaternions and Dual Quaternions) and Twist-And-Bend, along with a little geometrical/algebraic assistance for folks
like me who could use some help with matrix algebra.
- Robotic control ideas. These are the idle
speculations of an armchair computer scientist, maybe someone
will take them up and prove me wrong, or right. If I have time
and bandwidth, I will, but the lack won't justify a deepfreeze
for these specs.