What is the best programming language to implement neural networks?

Question

I'm not looking for a Neural Networks library, since I'm creating new kinds of networks. For that I need a good "dataflow" language.

Of course you can do this in C, C++, Java and co. but dealing from scratch with the multithreading etc. would be a nightmare.

At the other extremity, languages like Oz or Erlang seem more adapted, but they don't have many libraries, and they are harder to master (it's easy to play with them, but is it OK to create complete software ?).

What would you suggest ?

This link might be helpfull. http://www.trapexit.org/Erlang_and_Neural_Networks — Prashant Sharma, Oct 13 '12 at 14:02

score 15 · Accepted Answer · edited Jul 13 '12 at 14:05

15

I watched an interesting conference presentation about using Erlang for Neural Networks. You might want to check it out:

From Telecom Networks to Neural Networks; Erlang, as the unintentional Neural Network Programming Language

I also know that the presented system is going to be open-sourced any day now according the authors tweet.

edited Jul 13 '12 at 14:05

mmdemirbas

8,486
5
44
51

answered Jul 01 '11 at 14:46

alavrik

2,121
12
16

Yes thanks, I've seen it too, very interesting. I might go with this solution, Erlang seems quite mature now. My only concern with this solution is the complete asynchronousness of the network, you have no way to control any "timing". – Blacksad Jul 01 '11 at 22:40
In Erlang, you can deal with time and synchronization very easily. Much easier that in any other practical languages. So even if the basic system doesn't support that out of the box, it may not be very hard to extend it. Erlang is great for doing concurrent soft real-time systems by design. Besides telecom, there are other classes of systems that rely on this property, including trading, CEP, etc. – alavrik Jul 02 '11 at 00:01
1

Luopan, accept this answer if it is the best solution for you – Muzaaya Joshua Jul 02 '11 at 09:01
it is easy to deal with synchronization in such a network. Along signal (as a part of message) you can keep 'generation' of the message. You can also maintain 'tick' messages. Yet another approach would use states machines consuming just one signal and handling sync in some event way. – user425720 Jul 03 '11 at 00:15
Look at Gene Sher's post below, for a direct link to the library referenced above. – Charlie Jan 19 '15 at 22:42

score 10 · Answer 2 · answered Jul 03 '11 at 00:12

Erlang is very well suited for NN.

Neurons can be modeled by processes (no problem with having millions of them)
Connections/synapses can be represented by PIDs of target neuron. It is very easy to initialize such a network as part of standard init procedure in OTP. Communication would be realized by message passing.
Maybe it would be good to have global address space in ETS/mnesia (build in datastores) in order to do dynamic reconfiguration of network structure.
Pattern matching in receive block can determine what kind of signal neuron receives and modify it on the fly.
It would be very easy to monitor such a network.

Also consider that Erlang NN would be 'live' all the time. You would be able to query neurons, layers, routers etc any time. In C/C++ you just read current state of arrays/data structure.

Regarding performance, we all know that C/C++ is orders of magnitude faster than Erlang, however NN topic is tricky.

If the network would hold very few neurons, in very wide address space, in regular array, iterating over it again and again could be costly (in C). Equivalent situation in Erlang would be solved by single query to root/roots (input layer) neurons, which would propagate query directly to well addressed neighborhs.

The difference in performance between C/C++ and Erlang is **VERY** test dependant. For small tests like incrementing the fields in an array C/C++ is definitely much faster than Erlang, but for large, real applications the difference will be very small, if there is any at all. Complexity tends to favour those systems which are designed to handle it, like Erlang. — rvirding, Jul 03 '11 at 22:05

score 6 · Answer 3 · answered Jun 29 '13 at 18:57

6

DXNN1, and DXNN2 which was built and introduced in the textbook: Handbook of Neuroevolution Through Erlang: http://www.amazon.com/Handbook-Neuroevolution-Through-Erlang-Gene/dp/1461444624/ref=zg_bs_760204_22

Are open source and available at: https://github.com/CorticalComputer

answered Jun 29 '13 at 18:57

Gene Sher

61
1
1

The link in the post above is the open source library that alavrik was writing about. – Charlie Jan 19 '15 at 22:41
1

Extra points for probably the author himself giving the answer? – Chiffa Oct 12 '15 at 06:25

score 3 · Answer 4 · edited Dec 17 '12 at 10:28

3

If you are interested in data flow programming and multi-threading then I would suggest National Instruments LabVIEW. In this case you don't need to bother about multi-threading since its already there and you can also use OOP since now OOP is also native with LabVIEW. LabVIEW OOP is also purely based on data flow programming paradigm.

edited Dec 17 '12 at 10:28

Alexis Pigeon

7,054
11
36
44

answered Dec 17 '12 at 10:07

Mani

31
1

score 2 · Answer 5 · answered Jul 07 '11 at 05:07

If you have any Java experience, then use Scala which is a JVM language that is based on the same concept of "actors" as Erlang. But it is less strict than Erlang and can easily use any existing Java libraries.

Then, when you find a computationally expensive task that would work better in Erlang, you can use Erlang's jinterface library to communicate between your Scala code and your distributed Erlang nodes.

score 1 · Answer 6 · answered Dec 27 '11 at 03:28

1

Using Java does not mean dealing from scratch with multithreading - just use one of numerous Java Actor Libraries.

answered Dec 27 '11 at 03:28

Alexei

11
1

score 0 · Answer 7 · answered Aug 08 '15 at 23:41

0

Another big plus for Erlang is full integration with Drakon

http://drakon-editor.sourceforge.net/drakon-erlang/intro.html

answered Aug 08 '15 at 23:41

Gareth Thomas

410
3
3

Mitch Wheat · Answer 8 · 2011-07-01T12:14:52.130

0

Why reinvent the wheel? Try PyBrain. It's free and very comprehensive:

Quickstart

edited Jul 01 '11 at 12:14

answered Jul 01 '11 at 10:52

Mitch Wheat

280,588
41
444
526

Thanks ! But I'm trying to avoid (for performance reasons) my old love Ruby, so I cannot cheat her with Python :) – Blacksad Jul 01 '11 at 22:41
@Luopan : I don't see that fact mentioned in your question. Why limit yourself with arbitrary criteria? – Mitch Wheat Jul 02 '11 at 00:52
2

You also can try [Peach](http://code.google.com/p/peach/), both Peach and PyBrain are developed with the concern of performance (Peach is based on NumPy, that is implemented in C), and both are used successfully in scientific (with a lot of processing) applications. – renatopp Jul 03 '11 at 14:48

jonsca · Answer 9 · 2011-07-01T11:03:29.050

0

It's not a language in and of itself, but Emergent is very powerful and can be highly customized (it has a full scripting language).

It's open source, too, which could be helpful as a guide if you need to make your own version for your novel architectures.

edited Jul 01 '11 at 11:03

answered Jul 01 '11 at 10:55

jonsca

9,342
26
53
60

score -1 · Answer 10 · answered Jul 06 '17 at 08:22

-1

It all depends on your application. C++, Python are some good programming languages for machine learning

answered Jul 06 '17 at 08:22

Amy segenuelle

31
7

What is the best programming language to implement neural networks?

10 Answers10