Implement a C# interface to htm.core

[dkeeney]

This is a place to re-start discussions about a C# interface to htm.core.

[dkeeney]

I am thinking that the C# interface should be similar to the Python interface.

• There will be a set of bindings for C# written in C++/cli managed code that are linked with the the core library. This will be a shared library.
• There can be two types of interfaces.
  -- Those that directly call the algorithms in the C++ library. For these we will need a C++/cli wrapper for each of the algorithms.
  -- Those that use the NetworkAPI to access the algorithms. For this we will need one C# wrapper for the NetworkAPI and C# apps would then have access to every algorithm for which a Region wrapper has been written for it, regardless of the language it was written in.
  -- Any C# modules that implement an algorithm can also implement a NetworkAPI wrapper (which we call a Region). These regions are implemented as a separate plugin. The plugin mechanism would then allow C++ and Python apps to also access the C# algorithms (at least that is the theory).

[dkeeney]

Let me start by creating a Pull Request and I will add the code I have into it.
Then we can discuss the details of our new C# bindings.

[mrpitts52]

@dkeeney is the plan to merge your PR into master, or to continue working on that branch?

Also, down the road when the C# interface is ready, will it be possible to build both the Python interface and the C# interface in the same location? I'm wondering if it makes sense to have 2 separate local copies of the repo if I want to be able to use both. Thanks!

[dkeeney]

@mrpitts52 welcome to htm.core. I was hoping that you would join us.

is the plan to merge your PR into master,

The way this works is that we clone the repository and check out the branch (or create a new one) for each project. We work on the branch until we are satisfied with the code and it passes all of the unit tests locally. For any new code we must include new documentation (usually comments in code) to explain how it works and new unit tests to prove that it works as advertised. The unit tests are also ran automatically when we push a PR back to gitHub. When it passes all unit tests on all three platforms in the CI (Windows, Linux, OSx), we set the status on the branch to 'ready' in GitHub. Then it is peer reviewed by other developers (probably @breznak and @ctrl-z-9000-times ) and anyone else that is interested. If everyone likes it someone with write permissions can approve it and then we can merge it into the master. At that point it live and is available for everyone to use. Periodically we do a formal release and increment the version number... that is whenever we feel like we need one.

I have about 5 clones of htm.core on my machine, one for each project I am working on but I guess you could do it with one clone and switch between branches. Sometimes I will experiment with several different ways of doing the task and have a different clone for each. Then decide on the one that seems to work the bests...or abandon it altogether.

The current state of this branch is that it will not even compile. It is a bunch of code I wrote up about 1-1/2 years ago when I was thinking about this. It was not complete even then. Since then the library has changed quite a lot..for the better I think. I created a work-in-progress PR so you can have access to the code. We can use this as a starting point for the C# interface or we can throw it all out and start from scratch if you want. In fact, we might want to think about how to break it up into smaller parts so there is less to review at one time.

will it be possible to build both the Python interface and the C# interface in the same location?

We should be able to build Python, C#, and C++ with the same CMake build. However it should also be possible to only build one of the three using option flags for those folks that may have the other environments installed. C# is primarily a Windows thing so Linux or OSx folks may not want it.

[dkeeney]

@mrcslws I will let you drive this project. I am here to answer questions and explain how the interface needs to work - or you can tell me how you think it should work. If there is a part of it that you do not feel comfortable with you can pass it to me. Once we have a plan, and know what all of the parts are and who is doing what, we can go build it.

You said you were a C# developer. That is cool. Just so I know where you stand;

• Do you feel comfortable with C++/cli ?
• Do you have experience with standard C++ on multiple platforms?
• Do you have any experience with plug-in interfaces?
• Do you have experience with Python?
• About how much time are you expecting to spend. (I am retired and have lots of time but some poor folks have to hold down day jobs and have families to maintain).

[dkeeney]

oops @mrcslws, I was targeting @mrpitts52

[mrpitts52]

@dkeeney that sounds great.

• Do you feel comfortable with C++/cli ?

I actually don't have any prior experience with C++/CLI, but moving between languages hasn't been an issue in the past so it's a challenge I'm happy to tackle.

• Do you have experience with standard C++ on multiple platforms?

This one's a no, the cross platform stuff will be all new to me.

• Do you have any experience with plug-in interfaces?

Not with exactly what appears to be going on here. I do have experience setting up network APIs, so hopefully some of that experience helps.

• Do you have experience with Python?

Some, it's not a language I've used extensively in the past but I have strong backgrounds in several OO languages so I'm not too worried about this.

• About how much time are you expecting to spend

My full time job is unfortunately the top user of my time, along with other random obligations. That said, I'm excited to get into this, so hopefully I can give a couple evenings a week and some good chunks of time on the weekend. I wish I could do more but I'll definitely do what I can.

As mentioned before I have used C# quite a bit and I am very comfortable with git, so at least there a couple things here I do have a background in. If after reading all that you're still willing to let me drive this particular item I'd be happy to do so. 😃

[mrpitts52]

I just cloned the repo the other day and got everything building successfully. The readme is great for setup and some info on running test and examples. Is there anywhere in the docs that gives kind of a high level architectural overview, both for the core code and the existing Python interface? A basic overview that indicates what the various 3rd party dependencies are used for and maybe a general guide to what classes do what would be very helpful.

I've been dropped into code bases where I had no idea what was going on before, so I can work my way through the code and figure it out, but I would definitely befenit greatly from an overview like that to speed the process up.

[breznak]

Hi Mike, great to see another soul joining us!

I just cloned the repo the other day and got everything building successfully.

I'm happy the process worked smoothly for you.

Is there anywhere in the docs that gives kind of a high level architectural overview, both for the core code and the existing Python interface?

code-wise, or in generel HTM-wise? Actually, I think we're lacking on the general "what the heck is an HTM and why is it so good then?"-side. Not that we should do all that, but we should like to introductory papers and forum posts to help get people into the picture.

Is there anywhere in the docs that gives kind of a high level architectural overview, both for the core code and the existing Python interface? A basic overview that indicates what the various 3rd party dependencies are used for and maybe a general guide to what classes do what would be very helpful.

this is a good point, thank you! 👍 It would be nice to have an introductory readme for developers wishing to jooin and who need to understand the codebase.

a high level architectural overview

the main classes are

• Spatial pooler (SP)
• Temporal memory (TM)
• then Encoders, and Anomaly

I'd suggest looking at py/htm/examples/*, then src/examples/ (for c++), and then read through the unit tests for select classes of interest, some tests even have a "BasicUsage" example test.

what the various 3rd party dependencies are used for

I don't think that's needed for intro (?), unless you're looking for working on a very specific feature. Here, 3rd party are really only used as dependencies.

maybe a general guide to what classes do

again, the in-class source code doc is very good, but as a document we don't have any.

hope to help somewhat and ask away as you dig into the code :) (maybe start a new issue for the docs)

[dkeeney]

Ok, as he said, we don't have good overview documentation. But there are some things you can look at to get some idea of what is going on.

@breznak feels that the in-code documentation is adequate. And for figuring out the calling arguments for the algorithms I agree. But for overall understanding I would start with HTM School. Start at the first video and work down through them. It will take a while but it helps a lot in understanding the scope. Jeff Hawkins book On Inteligence is also a good read to put everything in perspective.

Then, there is a set of old documentation for the NuPic API. Parts of this are not accurate. The OPF Framework is no longer supported. The rest of it will give you some introduction to the terms used but we have diverged from this somewhat.

There are the main classes of the algorithms, Spatial Pooler, Temporal memory, Anomaly, and all of the encoders. which were mentioned by @breznak. Those are what the library are all about and everything else are support. For many people this is all they need. They can call the main algorithms from their apps and do whatever they need. In general, @breznak and @ctrl-z-9000-times address those modules.

But there is another set of modules which we now call NetworkAPI. These consist of the Network class and the Region class with its subclasses that make up the plug-ins. This is basically a wrapper around the algorithm classes which orchestrate the execution and data flow for applications. These are the classes that I concentrate on. This is an alternative way to execute the algorithms.

There is also a set of code that make up the C++ to Python interface. Calls actually flow in both directions to make things interesting. There is one set of interfaces to allow python apps to call the algorithms directly. There is another set of interfaces to allow python apps to use the NetworkAPI. The NetworkAPI class will also call back into Python to execute Regions that were written in Python.

The C# project will involve duplicating the Python interface for use by C# applications. Both the algorithm access interface and the NetworkAPI interface. Once those are in place you are free to build wild and wonderful things in C# using the htm.core library.

[dkeeney]

One of my TODO list items is to re-write the old NuPic API for our library as it is today. Perhaps not all at the calling argument detail but at least much of the overview. I hope someone would show up that has a passion for documentation and can help us out. :-)

[mrpitts52]

Thank you both so much!

The good news is that I have read On Intelligence and I've watched all the HTM School videos, but looks like there is still plenty for me to go through both in the code and over on the HTM Forum. By the way I'm mtopper over there, I like to have different IDs to be unnecessarily confusing.

As I go through the code and inevitably have questions, what's the most logical place to post those? I'm ok with continuing to use this thread but I understand if we'd rather avoid cluttering up an issue thread with a bunch of random questions.

I hope someone would show up that has a passion for documentation and can help us out

@dkeeney I was about to comment that documentation should be the favorite part of every developer's job 😆 I will open up a new issue for that, someone has to be the downer who brings it up 😃

[dkeeney]

I will open up a new issue for that,

We already have one.... Isssue #325