# Neural Network Studies

by Tom Veatch

| NN Top | Introduction | Newton's Method | Fuzzy Logic NNs | & training rule | Twist-and-Bend | Space Representing NNs | Dual Quaternions | Algebroids | Robots |

Neural Networks are an important idea of our time:

• An introduction to neural networks should enable any motivated high school graduate to understand them completely and correctly, as well as develop appropriate intuitions about what is going on inside them as it processes inputs and as it gets trained. I offer you this accessible mathematical introduction to NNs.

• Why don't folks use "Newton-Raphson", better named Successive Linear Approximation as the update rule for NN training? Instead, they say, crawl off in the right direction: does crawling make sense to you when you could jump most of the way there in one step? With an algorithm we charitably call Newton's method, we should rather make a good-to-excellent guess how far to go (namely, $$1/e''$$ times the slope $$e'$$). In case this is because folks think the math might be hard or unknown for Neural Networks, I offer you the derivation, along with arguments for its likely superiority. Now there's no excuse not to try it.

• Fuzzy logic is logic where truth isn't just 0 or 1 but any number in the range of [0..1]. In Neural Networks as Fuzzy Logic Engines, it is pointed out that neural network nodes can be interpreted as Fuzzy Logic Predicates (with outputs in the range [0..1]). Then the idea is contemplated and advanced that they can be re-engineered to learn Fuzzy Logic relationships so as to actually be, learning fuzzy logic engines. Neural Nets are enhanced by the semantic interpretability and the logical operators of Fuzzy Logic; while Fuzzy Logic benefits by the data-driven learnability of Neural Networks. FLENNs!

• Thinking about spatial perception as sound, or rather as echoic, two-signal sound, the ideas came to me of Space Representing Neural Networks. Inspired by my experience, once upon a time, being electrocuted (no not to death), and "seeing" the discharge path inside my body, this is one explanation of how an organism could have such an experience. An SRNN is a multi-dimensional, multi-scale, bi-directional, space/time representation and transformation system which enables capture and combination of information regarding position, size, shape, movement in merged perception from multiple sensors as well as the reverse, namely, self-assigning, choreographed activation trajectories for actuator-driven movement within space as a representational target for action planning, learning, and control.

• Three ways to implement the geometric transformations needed in SRNNs are offered in Dual Quaternions (namely Quaternions and Dual Quaternions) and Twist-And-Bend, along with a little geometrical/algebraic assistance for folks like me who could use some help with matrix algebra.

• Robotic control ideas. These are the idle speculations of an armchair computer scientist, maybe someone will take them up and prove me wrong, or right. If I have time and bandwidth, I will, but the lack won't justify a deepfreeze for these specs.

Another introduction to this work is here.

| NN Top | Introduction | Newton's Method | Fuzzy Logic NNs | & training rule | Twist-and-Bend | Space Representing NNs | Dual Quaternions | Algebroids | Robots |

 Your thoughts? (will not be shared or abused) Comment: Feedback is welcome.