top 200 commentsshow all 308

[–]perturbater 119 points120 points  (26 children)

Developing a language from scratch has got to be difficult for a number of reasons, but one of them is how you're forced to make design decisions without knowing the full ramifications for years. In hindsight, what were the best and worst decisions you've made?

[–]jeffbezanson 81 points82 points  (17 children)

This is one of the main reasons we spent several years making "experimental" 0.x releases. Over a year or so, we'd collect experience about what worked and what didn't, then make breaking changes --- sometimes lots of them. We did 7 cycles of that until we felt we really had a handle on things.

Best decision was to make a multiple-dispatch-based language. For a brief period in the beginning that wasn't 100% clear. Multi-methods and external dispatch seemed cool, but we had no idea just how incredibly useful and powerful they would be.

Worst decision: make all text in the terminal bold :-P Thankfully we fixed that.

[–]StefanKarpinski 22 points23 points  (15 children)

Best decision was to make a multiple-dispatch-based language. For a brief period in the beginning that wasn't 100% clear. Multi-methods and external dispatch seemed cool, but we had no idea just how incredibly useful and powerful they would be.

I cannot underscore how true this is. Taking multiple dispatch seriously has changed the way we program; it's a weird thing because when people ask "what's so great about multiple dispatch?" it's hard to pinpoint anything specific but then once you become used to it, you can't go back. I tried writing some Ruby some years ago, which I had done a lot of previously, and when I realized I couldn't write separate methods for different combinations of argument types, I just couldn't so I switched to doing it in Julia. That was actually the first iteration of our package manager. No regrets about writing it in Julia :)

[–]dipolartech 12 points13 points  (10 children)

So from your description, you have methods that consume different parameters, and this is different from overloading and just checking parameters in the method how?

[–]StefanKarpinski 14 points15 points  (7 children)

Compile time versus run time. It's the difference between virtual and non-virtual methods in single dispatch object-orientation but multiplied out by all the arguments instead of a single special one.

[–]fully_strapped 6 points7 points  (0 children)

Multiple despatch was an awesome decision, I live it & love it every day

[–]loladiro[S] 22 points23 points  (0 children)

Sometimes I'm surprised how much a seemingly minor features changes the way the language is used. At the hackathon after the first JuliaCon, Tim Holy and I implemented a feature called generated functions (staged functions at the time), which are basically a hook back into the compiler that allows you to generate code based on the types of the arguments. I wanted it for my C++ package (basically allowing julia to act as a C++ REPL) and he wanted it to speed up arrays. Nowadays, it's not used as much for arrays anymore, since the compiler has improved enough to make it unnecessary, but it is the basis for https://github.com/jrevels/Cassette.jl, which is a general non-standard executing framework that we're planning to use for everything from debugging to automatic differentiation.

[–]alanedelman 19 points20 points  (0 children)

this may not be the best or worse, or even the most important, but it did turn out that taking vector transposes seriously was a bigger problem than most anyone realized: https://github.com/JuliaLang/julia/issues/4774

[–]ViralBShah 10 points11 points  (5 children)

Just as everyone has pointed out, multiple dispatch was clearly the right decision. In hindsight, I do feel we should not have baked so many C++ and Fortran libraries into the Julia repo. What was done as a convenience early on, has proven to be hard to undo. Having an `stdlib` greatly helps now, and also makes it possible to have pure Julia versions. Maybe eventually we will have a Julia BLAS.

[–]StefanKarpinski 9 points10 points  (1 child)

OTOH, if we hadn't done that, we never would have gotten anywhere, so I think it was unavoidable.

[–]vks_ 1 point2 points  (0 children)

An alternative would have been to move them out of the standard library into external packages.

[–]ontouchstart 2 points3 points  (0 children)

Good point about the repo size.

I am building Julia from github master as a Docker image for portability and reproducibility. The compiling of julia-deps would take very long time every time I pull the master and build from scratch. Now Juila has reached stable 1.0, any plan to reduce the build dependency and runtime dependency? As a start perhaps the julia team can provide an official stable julia-deps base image so we can build upon.

[–]willis77 57 points58 points  (4 children)

It's too common that open source developers don't get the appreciation they deserve for what can be thankless work. So, firstly, thank you for your work on Julia!

Question: years ago, it felt like Julia was on a warpath to take on R and Python as the language to use for data science. It feels like that momentum has slowed lately. Do you agree and, if so, what do you think needs to happen to rekindle that enthusiasm?

[–]loladiro[S] 29 points30 points  (3 children)

I wouldn't characterize us as ever having been on a warpath with R or Python. We play very well with them through integrations like PyCall.jl and RCall.jl, as well as working with Jupyter notebooks. I think there's more than enough space for everybody.

With respect to data science, I agree that it's been lagging a bit behind in Julia compared to some other numerical application domains, but particularly with 1.0, we've tried to do a lot of the ground work to change that. With 1.0, we have compiler support for high-performance missing value support (https://julialang.org/blog/2018/06/missing), which was one of the biggest asks of the Julia data science community and I do hope they take good advantage of it, because it was a massive amount of work ;). On the package ecosystem side, meta-packages like the https://github.com/queryverse/Queryverse.jl are starting to make sure that all the disparate functionality is available in one uniform package. There's also https://github.com/JuliaComputing/JuliaDB.jl, which you can think of as a distributed, out-of core alternative to pandas, if you happen to have very large data sets. With all that, I do think the future is very bright here, and especially now that the core language has stabilized, we can focus a lot more attention on the core ecosystems.

[–]ViralBShah 15 points16 points  (0 children)

Yeah, we never thought we are on a warpath with R and Python. All of us routinely go to various R and Python conferences, and I also had the fortune of attending an R committee meeting a couple of years ago. It has always been about making progress and doing new things, rather than trying to replace something that works well.

[–]count___zero 29 points30 points  (5 children)

Do you think in the future Julia could take the place of Python for scientific computing?

I am doing machine learning, mainly programming with Python. I tried Julia a year ago and I liked it, but the problem is that I invested a lot in my experience with Python and its ecosystem of libraries (and I also hate the MATLAB operator for the transpose).

How do you plan to convince people to switch to Julia? Are you going to port popular libraries like sklearn and pytorch?

[–]jeffbezanson 30 points31 points  (3 children)

I just keep thinking of Steve Martin's "be so good they can't ignore you".

On the ecosystem side, a key advantage of julia is that libraries are written in pure julia, not in C and C++. We get leverage from that. Developing flexible, high-performance libraries is just wildly easier in julia, so "catching up" in terms of functionality starts to seem realistic.

[–]StefanKarpinski 29 points30 points  (2 children)

A few years ago I don't think anyone would have believe that Julia's package ecosystem would be where it is today. There's a number of areas where it's hands down better than anything else—optimization, differential equations, linear algebra, numerical analysis (too many packages to even link a single one), among others. And the same momentum that has allowed us to catch up and surpass other ecosystems in such a short time continues.

Julia already has OpenMP-style for loop multithreading, but we're nearly ready to merge a pull request that implements M:N mapping of tasks onto hardware threads, which will make Julia's threading system similar to Go's but tuned for extreme computational performance rather than for writing concurrent servers (although you can do that as well). As we see the number of cores on CPUs go up and the amount of memory per core go down, this will increasingly be a huge competitive advantage over other dynamic languages that don't have any plausble answer to multithreading.

[–]harponen 1 point2 points  (0 children)

I think by machine learning /u/count___zero meant computing on the GPU. I'm quite certain that scientific computation in general will move heavily towards GPU computing (or that at least popularity of ML and neural networks will continue their aggressive growth). I would like to know how important you regard these development directions.

[–]ViralBShah 8 points9 points  (0 children)

Projects like Celeste.jl are what I think will eventually make the scientific community use Julia in the long run for high performance programs.


Cataloging the Visible Universe through Bayesian Inference at Petascale


[–]NotSoGreatLeader 59 points60 points  (9 children)

As a CS student, can you give me reasons to learn Julia instead of any other language already in use in the industry?

What type of projects should I do with it?

[–]cormullion 15 points16 points  (5 children)

You might enjoy this video from the recent Julia conference about systems programming

[–]jeffbezanson 35 points36 points  (4 children)

To expand on that, I'd say that julia gives you access to an unusually wide range of the programming stack. You can manipulate bits and bytes with good performance (and see the assembly code for it!), but then also do higher-order programming, macros, and object-oriented design and it all fits together. It's easy to peel back the layers. People often switch languages to do different kinds of tasks. Julia programmers use different techniques and styles for different tasks, but staying within the same language.

[–]StefanKarpinski 47 points48 points  (3 children)

and see the assembly code for it!

Of course, this is true in any compiled language, but in Julia it's really easy:

julia> f(x, y) = 2x^2 + 3y - 1
f (generic function with 1 method)

julia> f(4, 5)

julia> @code_native f(4, 5)
    imulq   %rdi, %rdi
    leaq    (%rsi,%rsi,2), %rax
    leaq    (%rax,%rdi,2), %rax
    addq    $-1, %rax
    nopw    %cs:(%rax,%rax)

That easiness makes all the difference—I almost never bother looking at the assembly code in C because it's such a pain to do. I look at Julia assembly code all the time because it's so easy and I know lots of others do as well. This is both amazing as a learning tool and as a way to guide the entire ecosystem towards great performance.

[–]matthieum 9 points10 points  (1 child)

That's a really neat feature of the Julia REPL indeed.

For C (and others), there's the amazing Compiler Explorer (godbolt), which is pretty cool for that. It makes it easy to switch compiler versions and flags to check the effect of optimizations, and has highlighting to link together the source code and assembly instructions... but it does require extracting a minimal example from whatever code you're working on and pasting it into the browser, then after fiddling pasting the code from the browser back into the project.

[–]StefanKarpinski 9 points10 points  (0 children)

godbolt is awesome, we use it all the time to compare what Julia generates with what various C compilers do.

[–]Elronnd 2 points3 points  (0 children)

FWIW, I never found it that hard to look at disassembly in c (can run objdump, or gdb+disas). You can also do a similar sort of easy disassembly in common lisp:

* (disassemble '(lambda (x y) (+ (* 2 x x) (* 3 y) -1)))

; disassembly for (LAMBDA (X Y))
; Size: 101 bytes. Origin: #x10034A6BAB
; BAB:       488B0C2508041020 MOV RCX, [#x20100408]           ; no-arg-parsing entry point
                                                              ; SB-VM::*BINDING-STACK-POINTER*
; BB3:       48894DF8         MOV [RBP-8], RCX
; BB7:       488B55F0         MOV RDX, [RBP-16]
; BBB:       BF04000000       MOV EDI, 4
; BC0:       FF1425B800B021   CALL QWORD PTR [#x21B000B8]     ; GENERIC-*
; BC7:       488B7DF0         MOV RDI, [RBP-16]
; BCB:       FF1425B800B021   CALL QWORD PTR [#x21B000B8]     ; GENERIC-*
; BD2:       488BDA           MOV RBX, RDX
; BD5:       48895DE0         MOV [RBP-32], RBX
; BD9:       488B55E8         MOV RDX, [RBP-24]
; BDD:       BF06000000       MOV EDI, 6
; BE2:       FF1425B800B021   CALL QWORD PTR [#x21B000B8]     ; GENERIC-*
; BE9:       488BFA           MOV RDI, RDX
; BEC:       488B5DE0         MOV RBX, [RBP-32]
; BF0:       488BD3           MOV RDX, RBX
; BF3:       FF1425A800B021   CALL QWORD PTR [#x21B000A8]     ; GENERIC-+
; BFA:       48C7C7FEFFFFFF   MOV RDI, -2
; C01:       FF1425A800B021   CALL QWORD PTR [#x21B000A8]     ; GENERIC-+
; C08:       488BE5           MOV RSP, RBP
; C0B:       F8               CLC
; C0C:       5D               POP RBP
; C0D:       C3               RET
; C0E:       CC0F             BREAK 15                        ; Invalid argument count trap

[–]DiogenicOrder 20 points21 points  (7 children)

Is there a plan to create a consistent environment for the machine learning and AI packages?

Maybe something like sklearn for Python?

For now the ecosystem is still scattered around and most packages aren't working with Julia 1.0

What's the roadmap, the projects, how do you envision the near future?

[–]ViralBShah 29 points30 points  (2 children)

The community is quite focussed on having all of Julia be naturally useful for AI. The focus is not so much to create yet another framework, but to make sure that the whole of Julia is available for machine learning. This pretty much boils down to two things: having great support for Automatic Differentiation (AD), and having good support for native code generation on hardware accelerators (Mostly GPUs, but also increasingly TPUs and various new things in the pipeline).

At JuliaCon, Jarrett Revels announced Cassette.jl (https://github.com/jrevels/Cassette.jl) and Capstan.jl (the AD package that leverages Cassette's compiler enhancements). With these packages, we now have a general way to do AD on entire Julia programs. The CUDANative.jl and related GPU packages give us a general way to run Julia on GPUs, and the underlying refactoring makes it easy to target TPUs and other special purpose processors. With all these projects stabilizing for 1.0, we feel that Julia is already a compelling language for AI researchers and users.

In the meanwhile, we have Flux.jl (https://github.com/FluxML/Flux.jl) by Mike Innes, and Knet.jl (https://github.com/denizyuret/Knet.jl) by Deniz Yuret, both of which provide significant AI capabilities. The goal is to have Framework-less AI. Just write straight Julia code, and we'll be able to differentiate it, stick it into an optimizer, run it on a GPU - without needing new programming models (like writing the computational graph) or frameworks that re-implement all the libraries.

[–]Bdamkin54 9 points10 points  (0 children)

That's great for custom algorithms, but there is a still a significant place for framework facilitated canned non diffable algorithms, model selection/eval and other aspects of more mundane data science. For Julia to gain mainstream data science appeal, it will have to approach and exceed scikit learn for these sorts of tasks. (and I think it will soon, given the way things are moving.)

[–]DiogenicOrder 1 point2 points  (0 children)

thanks for your time!

[–]jeffbezanson 17 points18 points  (3 children)

most packages aren't working with Julia 1.0

Right now this is true, but it's changing rapidly. We're seeing dozens of packages get fixed per day, sometimes as many as 100.

[–]DiogenicOrder 6 points7 points  (2 children)

Great! I was hoping to start tinkering for real in September =)

Still, there are packages left and right which often do the same thing and some that have so little visibility that they're difficult to find, any chance there will be an overarching management to find what one looks for and not have dependencies breaking?


[–]StefanKarpinski 22 points23 points  (1 child)

Interestingly, in other, less experimental areas of computing, Julia has way better cohesion than other languages and ecosystems. Examples:

  • a single color types package that gets used anywhere anyone wants to work with colors
  • a single distributions package that gets used anywhere anyone needs to work with statistical distributions

The ability to reuse and share code so easily seems to stem from multiple and external dispatch. So there's no technical reason that AI/ML couldn't be as unified or more in Julia than it is in other languages, but I think that the field itself is still in an experimental phase where we don't know what "the right" way to do things looks like. As a result, you get a profusion of libraries and frameworks that are all experimenting with the best way to do ML. ScikitLearn is awesome if you're using relatively established, standard methods, but if you want the cutting edge in terms of performance and/or algorithms then you have to use other systems like TensorFlow or PyTorch, which are entirely disjoint from ScikitLearn. So the fracturing seems to me not to be specific to Julia but merely where the field of ML/AI is at right now.

[–]DiogenicOrder 1 point2 points  (0 children)

thanks for your time!

[–]joseph_miller 16 points17 points  (7 children)

I'm mostly an R programmer, but have been experimenting with and reading about Julia obsessively all year.

I've learned a TON about programming language design/theory from following discussion/blogs on Julia, reading the excellent documentation, and researching on my own.

Do you guys have any recommended books or resources on the subject to save myself and others some time? If you had to learn about programming language design from scratch, what would you recommend?

[–]loladiro[S] 15 points16 points  (1 child)

I kind of cheated by working with Jeff, but I think the best way to go about learning about programming language design is to use as many languages as possible. They each have their own feel, and figuring out what things work and why is crucially important. I did learn a little bit about formal programming language theory, but I haven't actually found it super useful in the day to day work of language development. A lot of what we do is API design, which is much more of a human question than a technical question (many APIs are technically possible, most are a horrible idea). Knowing what other things are out there that have solved the same problem is hugely beneficial for that.

In terms of implementation, basically everything I know about compilers has been a result of needing to do something and then reading up on the theory of how to do it. I'm much more of a "top-down" learner where I start with a problem and then recurse into the underlying concepts to figure out how to do it. E.g. I start by looking at what LLVM does, try to figure out why it does it and then read the papers it references.

There is of course various books that are often cited. The Dragon book (Compilers: Principles, Techniques, and Tools) is a classic. I found https://softwarefoundations.cis.upenn.edu/ a pretty good textbook for the more academic PLT view of things while still being practical enough to keep me interested.

[–]jeffbezanson 16 points17 points  (4 children)

The "90 minute scheme to C compiler" is a gem I always refer people to: http://churchturing.org/y/90-min-scc.pdf

In general it is worth learning scheme (which doesn't take long because it's very simple) and how to compile it (or compile *to* it) because it strips away all the unnecessary details and leaves only the most important core concepts.

[–]A_bottle_salmonella 12 points13 points  (7 children)

With all the current programming languages around, why do you think yours would be so much more groundbreaking?

[–]StefanKarpinski 32 points33 points  (6 children)

The main thing that makes Julia special is a deep commitment to multiple dispatch and performance. Multiple dispatch is a feature not many languages have and even the ones that do tend to relegate it to being a kind of fancy feature that you use on rare occasions but not in regular programming and it usually has fairly massive overhead. In Julia _everything_ uses multiple dispatch. Operations as basic as integer and floating-point addition and array indexing are defined in terms of it. The most ubiquitous, performance-sensitive operations go through the same machinery as user-defined types and operations. That means that you can define your own types and use multiple dispatch as much as you want without being afraid of the overhead. This is transformative in terms of how people use the language, and as a result, how the ecosystem develops. What we've seen as a result of this is that people define lots of very low-overhead types and use multiple dispatch all over the place. The effect is that different parts of the ecosystem compose really well and you get a multiplicative effect where the amount you can do is a product of all the components available to you instead of just the sum.

A really great example of this in action is the talk at JuiaCon this year by Robin Deits from one of MIT's robotics labs where all the different unrelated packages by different authors just composed to create this fully functioning robotics system. And then he just threw in as an aside that you can start with special measurement error values using the Measurements package and it just passes through all the layers, including rigid body dynamics calculations and differential equations solvers and in the end you get estimates of how accurate your computations are based on initial measurement error. And the kicker: the whole system is faster than real-time to the point where they have to insert sleep calls into the code to slow it down to sync up with real robotics systems.

[–]ViralBShah 16 points17 points  (1 child)

The performance especially makes it possible to have pure Julia versions all the way down, which is really valuable in machine learning.

Even 1+1 is effectively implemented in Julia itself. While not interesting to the end user by itself, having a pure Julia implementation of large parts of a user's program means that it is now possible to automatically differentiate it. It is now possible that Julia may end up becoming the first practical language for differentiable programming. One can compose differential equations with an optimizer, or incorporate physics into machine learning, or optimize mathematical functions that are difficult to do in other languages.

[–]matthieum 16 points17 points  (0 children)

The performance especially makes it possible to have pure Julia versions all the way down, which is really valuable in machine learning.

Besides machine learning, I would also expect to have a much better user-experience while debugging.

When a stack bounces back and forth between languages, the debugging experience is always spooky (short of debugging assembly). Often times, functions in one language are feature-rich, while functions in the other are black-box.

[–]MostlyFermions 3 points4 points  (1 child)

What is "Multiple dispatch" ?

[–]waldyrious 2 points3 points  (0 children)

For convenience, Robin Deits' JuliaCon 2018 talk is this one (video).

[–]bryanwag 9 points10 points  (2 children)

Do you think there is still a need for people to purchase Matlab license anymore when Julia can handle matrix operations just as well?

[–]loladiro[S] 20 points21 points  (0 children)

Depends on what you're doing. We're actually seeing more people migrate from bespoke legacy systems or C++ than from Matlab. Particularly if Matlab works fine for you, there's often very little reason to switch. If you were starting a new project today, Julia would probably be a good option, but there are other of course (Python chief among them). There's probably not a lot of reasons to start a new project in Matlab these days.

[–]adit07 21 points22 points  (3 children)

Is it ready for enterprise level deployment and scalability (supports parralel and distributed?) Do you have benchmarks on performance vs other languages? Thanks for hosting this ama.

[–]ViralBShah 19 points20 points  (2 children)

Yes, we believe it is. It has already been used in production by dozens of organizations. Benchmarks here: https://julialang.org/benchmarks/

This is a great keynote from JuliaCon about using and scaling Julia in production at Aviva, one of UK's largest insurers: https://www.youtube.com/watch?v=_jx1VmWxgVY

[–][deleted]  (1 child)


    [–]RoyiAvital 8 points9 points  (3 children)

    First, I would like to thank you to create such amazing project as Julia.
    For me it is an amazing experience see how this great language evolves.
    It truly shines because of you doing such a brilliant work with your choices and ambitions, it reflects your amazing capabilities.

    With your permission 3 questions:

    1. Could you describe your work day? How does it look like for Language developers?
      For instance, Keno, what does it means to be the main developer of the Julia Compiler?
    2. What's the next big decision, relating to Julia, on the table now? What are your thoughts about it?
    3. What's your second favorite Language after Julia?

    Thank You.

    [–]loladiro[S] 14 points15 points  (0 children)

    Could you describe your work day? How does it look like for Language developers? For instance, Keno, what does it means to be the main developer of the Julia Compiler?

    I think the workday is pretty much the same as any software developer, with the primary difference being how much of the communication is done out in the open. The start of my day is usually spent catching up on GitHub issues, email, discourse, etc. Afterwards, I usually either have a long-term ongoing project (for much of the past six months that was reworking the optimizer to handle the new iteration protocol and our new missing data support) for some high-priority bug that I need to take care of. Then there's various calls and meetings. For example, every Thursday, we have a "triage" call, where we discuss the most urgent items that need taken care of, what things are release blocking and generally try to make decisions on things that get stuck. These calls can be quite intense (on occasion they've gone on for six hours).

    What's the next big decision, relating to Julia, on the table now? What are your thoughts about it?

    There's lots of design decisions to be done every day, but I think the biggest upcoming area will the multi threading. We're adding support for scheduling co-routines onto multiple threads (https://github.com/JuliaLang/julia/pull/22631) and to take advantage of that well, we'll have to figure out various aspects of the parallel memory and execution model.

    What's your second favorite Language after Julia?

    I don't really pick favorites, but I think there's a couple of languages we can learn things from. For example, I'd really like to do something about restricted memory ownership models (which would allow more aggressive optimizations in various cases) and the Rust people have obviously done a lot of thinking there. We can't quite copy their design, since our users have different expectations, but I'm still hoping there's something we can do there. I'm also partial to formal verification like you'd find in Coq or Idris.

    [–]jeffbezanson 8 points9 points  (0 children)

    Thanks for the kind words.

    1. I start by looking over github, plus maybe slack and discourse, to see what's new and if anything really urgent has come up. Then there is a queue of issues I'm working on, and PRs to review. Sometimes that takes up the rest of the day, and sometimes I have time to work on longer-term things after dinner.
    2. What should we be working on for v1.1, and v2.0? All about prioritization.
    3. scheme

    [–]StefanKarpinski 5 points6 points  (0 children)

    (1) Pretty much the same as Jeff and Keno:

    • catch up on GitHub, Slack, Discourse
    • meetings whenever they are
    • fixing bugs/issues that are my responsibility
    • work on whatever long term project I'm working on

    Recently my project has been the new package manager and changes to the language's code loading required to support it. Before that it was overhauling strings so that they can read and hold arbitrary data including invalid UTF-8 while presenting a coherent, Unicode-compliant character model, without compromising performance at all.

    (2) Stopping and taking stock of where we are and deciding what's next. So that's kind of a meta task.

    (3) C.

    [–]sinsecticide 16 points17 points  (10 children)

    Hey all, thanks for all your amazing work on Julia! Any favourite packages? Packages that people should know about but haven't gotten the attention they deserve?

    [–]alanedelman 16 points17 points  (0 children)

    Interact.jl remains my favorite --- everything deserves an @manipulate.

    [–]ViralBShah 9 points10 points  (0 children)

    Generic Linear Algebra is a really cool package that has pure Juli implementations of various linear algebra algorithms, and can work with a variety of number types. Something you can't find easily in other systems.


    [–]jeffbezanson 8 points9 points  (0 children)

    It does get attention, but I'll mention it anyway: UnicodePlots (and ASCIIPlots)! Colors.jl is also mind-blowing. I don't think there's any other color-space library out there like it. Unitful and AxisArrays are also worth mentioning since they let you do things that are pretty difficult in most other languages.

    [–]nuclearfission 8 points9 points  (0 children)

    You might also want to check out juliaobserver.com.

    It tries to give you an overview of the julia package ecosystem (i.e. what's popular, trending, in machine learning, etc.)

    [–]StefanKarpinski 7 points8 points  (4 children)

    Packages that I always have handy:

    • Revise — so brilliant! Tim Holy is a real life superhero
    • BenchmarkTools — gotta measure things
    • StaticArrays — this should probably be a standard library
    • UnicodePlots — the only plotting package that reliably installs anywhere 😭

    That's about it, which is kind of a disappointing list, but I've spent so much time the past five years working on Julia itself that I haven't gotten much chance to use it in earnest to solve problems, but now that we've got 1.0 out maybe that will change. Then again, maybe not... we'll see.

    [–]ontouchstart 2 points3 points  (2 children)

    Any plan to add UnicodePlots or similar functionality to the stdlib REPL?

    [–]StefanKarpinski 3 points4 points  (1 child)

    It would be a good candidate for the standard library but we probably want to make stdlib packages more like regular packages going forward so ultimately it may not make much difference.

    [–]ontouchstart 2 points3 points  (0 children)

    Then how about bundling it with REPL itself? Doesn't matter where it lives.

    [–]loladiro[S] 3 points4 points  (0 children)

    Revise.jl has got to be my favorite package. OhMyREPL.jl is pretty awesome also.

    [–]soft-error 7 points8 points  (1 child)

    Is static compilation of Julia code/packages mature enough already for production?

    [–]loladiro[S] 9 points10 points  (0 children)

    It does work and people are using it (even in production), but it's a bit complicated at the moment. There's definite improvements planned to the implementation and the tooling in the 1.x development cycle. Some of the biggest things on the todo list here are to make sure that binary dependencies work well (https://github.com/JuliaPackaging/BinaryBuilder.jl will help) and to make sure we can produce binaries that only have the code you need for you application, rather than including everything under the sun. That said, you may be interested in the following videos from JuliaCon, which discussed how to ship statically compiled applications with Julia:



    [–]JayMoots 7 points8 points  (8 children)

    Where does the name come from? Is this a Facebook situation where one of you was dumped by a girl named Julia?

    [–]ViralBShah 9 points10 points  (6 children)

    The name Julia comes from an earlier project that Jeff had called Julia. In fact, in our keynote at JuliaCon (https://www.youtube.com/watch?v=1jN5wKvN-Uk), we discuss this in a bit of detail. Our very first email conversation had the name Julia already in it, and the julia logo was already in one of the first commits.

    [–]OvidiusCicero 3 points4 points  (2 children)

    Could give a timestamp for this? Watching The whole keynote seems a bit boring to me…

    [–]ViralBShah 4 points5 points  (1 child)

    9:50 into the talk.

    [–]Myopic_Cat 1 point2 points  (2 children)

    That begs the question: is there any particular reason why Jeff called his other project Julia? This isn't explained in the video either. Or is it just Julia all the way down?

    [–]StefanKarpinski 6 points7 points  (1 child)

    Or is it just Julia all the way down?


    • 🐢
      • 🐢
        • ...

    [–]loladiro[S] 17 points18 points  (0 children)

    julia> 🐢() = 🐢()

    🐢 (generic function with 1 method)

    julia> 🐢()

    ERROR: StackOverflowError:


    [1] 🐢() at ./REPL[1]:1 (repeats 80000 times)

    [–]jeffbezanson 10 points11 points  (0 children)

    We've made up various different stories. But the more I think about it this tweet got it right: https://twitter.com/fcollective/status/1027313313913495552

    Julia Child lived in the Cambridge, MA area. And I like to think that, the same way she brought good cooking to an American public that had been largely deprived of it, we try to bring good programming language techniques to kinds of computing that haven't seen enough of them. I also do enjoy her recipes. I make her champignons farcis fairly often :)

    [–]JayMoots 7 points8 points  (3 children)

    What is each of your "Silicon Valley" character counterparts?

    [–]StefanKarpinski 10 points11 points  (0 children)

    I won't speak for anyone else but I'm clearly Erlich Bachman

    [–]alanedelman 6 points7 points  (0 children)

    I am not sure about counterpart, but I do identify with getting lost in a data center. :-)

    [–]ViralBShah 3 points4 points  (0 children)

    I really love Erlich Bachman. But so does everyone else! And increasingly Jian Yang!

    [–]alan_du 7 points8 points  (1 child)

    One of Julia's key selling points is solving the two languages problem, where instead of writing my core library in C/C++ and my application in Python, I can do everything in Julia. In the meantime, however, it seems like we have a new two "devices" problem where now I write my core computing algorithms in CUDA for the GPU (or TensorFlow for the TPU) and wrap those in a higher-level API. This is especially acute in machine learning, where there a ton of different accelerators being developed (from GPUs to TPUs to FPGAs).

    What do you think Julia's role is in this new "two devices" world?

    [–]loladiro[S] 11 points12 points  (0 children)

    Write everything in Julia still ;). Here's our GPU compiler: https://github.com/JuliaGPU/CUDAnative.jl. I also spent some time at Google earlier this year to discuss running Julia on TPUs (put on hold to get 1.0 out, but I intend to take it up again). At the end of the day, everything has to be programmed in something and why not have that something be Julia.

    [–]battlehardenedguy 26 points27 points  (4 children)

    We’ve spent the past 9 years developing a new programming language.

    And how do you plan to break even?

    [–]ViralBShah 56 points57 points  (0 children)

    Julia is an open source project, so by itself, except for community resources, it does not have to worry about such things. Usually, we make a little profit from JuliaCon, which funds the community infrastructure like CI, downloads, etc.

    Julia Computing, on the other hand, employs many Julia contributors and does need to break even. We are fairly conservative as a business and have grown on the strength of customer contracts, government grants, and foundation funds. Investment by General Catalyst and Founder Collective made it possible for us to start thinking more in terms of a scalable product oriented business. Now that 1.0 is out, we are focussing on online training and JuliaTeam (to make it easy for corporate users to install, upgrade, and govern Julia+packages installations) to build a scalable product business.

    [–]jeffbezanson 18 points19 points  (0 children)

    Speaking personally, I already have. The project is its own reward :-D

    [–]alanedelman 18 points19 points  (1 child)

    Perhaps selling julia t-shirts ? :-) They are really popular!

    [–]ChrisRackauckas 20 points21 points  (0 children)

    And stickers. Everyone always asks for the sticker price (pun intended).

    [–]qKrfKwMI 9 points10 points  (0 children)

    This one is for /u/StefanKarpinski

    After having spent quite some time through the github issue tracker (which happens less now than it did a few years ago) and stackoverflow, I saw your profile picture so often that when I then finally saw a video with you in it I was a little bit disappointed that you weren't wearing a viking helmet like in the picture, so why don't you wear a viking helmet every day?

    [–]kbouser 5 points6 points  (2 children)

    I am just now looking at the Jane Herriman's intro video. Seems normal and expected for a language like Julia. However, I was wondering if Julia has Haskell-like pattern matching (on both left and right hand sides)? Have such features been discussed, rejected, tabled?

    [–]jeffbezanson 7 points8 points  (1 child)

    We don't have ML- or Haskell-like pattern matching exactly, but (1) multiple dispatch naturally gives you a bit of that, and (2) with macros it's possible to develop various kinds of pattern matching constructs, e.g. https://github.com/kmsquire/Match.jl . Whether pattern matching will be a core part of the language is an interesting open question.

    [–]kbouser 3 points4 points  (0 children)

    Thanks for the reply. We are so addicted to pattern matching that it has all but smoothed modern programming.

    [–]Polyfunomial 5 points6 points  (3 children)

    Are there ever any plans or dreams for a Julia CAS?

    [–]ViralBShah 5 points6 points  (0 children)

    There's this amazing project: http://nemocas.org/

    [–]loladiro[S] 3 points4 points  (0 children)

    We're not planning to work on one ourselves, but there's a group in Germany that has a fairly large grant to do exactly this: http://nemocas.org/.

    [–]Mr_Again 5 points6 points  (3 children)

    Just wanted to say I'm very excited for Julia and have been trying to get the jump on everyone else by using it early. Coming from python, i like the fact that there is a pythonic way to do things. Julia is much more flexible but is there or will there perhaps in the future be an equivalent concept?

    [–]loladiro[S] 5 points6 points  (0 children)

    Yes, people refer to a Julian way of doing things. Usually that means making good use of multiple dispatch or taking some of the patterns that have been successful in popular packages and using them in your own code. We don't want to ever require anybody to do something a certain way though.

    [–]StefanKarpinski 2 points3 points  (1 child)

    We already refer to this as "Julian" and there's definitely a strong sense of more and less Julian ways to do things. It takes a while when you start using the language to get the feel for it, but there does seem to be a convergence on Julian ways to write code and think about things, although, as with any language, it takes a while to get the hang of it.

    [–]Mr_Again 1 point2 points  (0 children)


    [–]OvidiusCicero 4 points5 points  (7 children)

    To u/loladiro (Keno Fischer), just out of pure curiosity: Where are you from? You seem to have studied in the US but I'm not sure whether you're American also.

    [–]loladiro[S] 15 points16 points  (6 children)

    I'm from Germany. Lived in Ratingen, near Düsseldorf for most of my life and went to Gymnasium (Germany middle school/high school) in Essen. I came to the US as an exchange student when I was 14 (fun fact, you can fly alone as a 14-year old on a German airline, but not on an American one and if you're booked on both, they will just leave you stranded in the middle) and spent two years in Salisbury, MD, before moving up to Cambridge, MA for college, where I've lived since.

    [–]StefanKarpinski 14 points15 points  (1 child)

    I came to the US as an exchange student when I was 14 (fun fact, you can fly alone as a 14-year old on a German airline, but not on an American one and if you're booked on both, they will just leave you stranded in the middle) and spent two years in Salisbury, MD

    This makes it sound like you got stranded in Maryland for two years because of an airline transfer issue 😂

    [–]loladiro[S] 5 points6 points  (0 children)

    Hah, I first got stranded by Lufthansa in Frankfurt, after being denied boarding to Philly. Luckily we managed to convince Lufthansa to put me on one of their flights to Philly instead and we got my host family to pick me up there (still a decent drive from Salisbury).

    [–]OvidiusCicero 1 point2 points  (2 children)

    What were you're reasons for pursuing a degree outside of Germany and even outside of Europe?

    [–]loladiro[S] 7 points8 points  (1 child)

    I didn't really plan to do go to college in the US, but I had been watching lecture videos on MIT OCW, so I was a bit obsessed with MIT being the place I wanted to go. That didn't end up working out, but during my time in Maryland, I did get the chance to go to Harvard, and it seemed to me that that was not an opportunity one should turn down.

    [–]OvidiusCicero 1 point2 points  (0 children)

    Interesting, thanks

    [–]chun4281 4 points5 points  (4 children)

    This is my first time hearing about Julia. I was wondering, who do you imagine would most use Julia and to Keno Fisher, I'm really curious, how did you get involved with this project as a high schooler?

    [–]loladiro[S] 9 points10 points  (3 children)

    Julia was created as a way for people who need to get numerical work done to do so faster, easier and in particular without having to switch to a different language, just because they wanted to get a bit of performance. As such, we've seen the most uptake in academia and in industries that need to do a lot of computing (e.g. Finance).

    As for how I got involved, I originally learned about this area of computing my taking Andrew Ng's ML MOOC, which was taught in Octave. Afterwards, I participated in a competition called the Mathematical Contest in Modeling (https://www.comap.com/undergraduate/contests/mcm/), where you basically model a problem and write a research paper about it over the course of 96 hours. The Octave code I wrote for that (to simulate the total mass of leaves on a tree), ran fairly slow, so I had to be awake every 3 hours to check the results, add them to the paper, tweak things and kick off another run. I was already a pretty good programmer, so I considered helping out with Octave to make it go faster, but a few weeks later Julia was announced, so I decided to work on that instead. One of the fantastic things about GitHub is that everybody can contribute, independent of your level of experience, where you come from or what your background is. The Julia community is very inclusive. I'm also happy to say that I'm not the only high schooler who's taken this path with the Julia Community. E.g. Yingbo Ma, who is one of the most prolific contributors started in high school.

    [–]pkofod 4 points5 points  (1 child)

    What are you looking forward to in v2.0?

    P.s.: Thanks for all the great work so far!

    [–]loladiro[S] 10 points11 points  (0 children)

    Figuring out the multi-threading model over the 1.x time frame should lead to it being amazing in 2.0. I'm also looking forward to getting better tooling. There's some amazing things we've been talking about for a very long time with respect to profiling, debugging, better error messages etc. Other than that, there's some amazing things happening in the ecosystem and I think the primary emphasis of 2.0 should be whatever support those packages need from the core language.

    [–]Mattiz 4 points5 points  (1 child)

    I've checked out the progress of Julia every once in a while. Now that 1.0 is out, what's the best way to learn the language and maybe even contribute to it?

    [–]loladiro[S] 3 points4 points  (0 children)

    Check out the youtube channel (https://www.youtube.com/user/JuliaLanguage) for some great introductory material on various topics. As for contributing, just pick anything on GitHub labeled with the "good first issue" label: https://github.com/JuliaLang/julia/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22.

    [–]coryrenton 4 points5 points  (2 children)

    what is the weirdest programming language that inspired you or that you borrowed features from? what features would you like to incorporate but can't because they simply wouldn't be accepted by the user base?

    [–]alanedelman 10 points11 points  (0 children)

    I loved APL, which some people find weird and unreadable, but it was pretty fun.

    [–]StefanKarpinski 10 points11 points  (0 children)

    I did a lot of Perl programming back in the day, which probably serves more as a "what not to do" example than anything else, in terms of language design, but one thing that's kind of hard to put your finger on that I always loved about Perl is that it's just FUN to use. People have told me that Julia is fun in the same way that Perl is. I think this has something to do with the language letting you do subversive things that might be dangerous or questionable but which work and let you play with the computer in interesting ways. Julia's range of different levels of programming from very low-level bit twiddling to very high-level dictionary slinging, and the tons of introspection and code generation tools, and the language never telling you that you can't do something (merely encouraging you not to do the really sketchy things) seems to combine into it being a really fun language. So if we got anything from Perl, I hope it's that.

    [–]Babahoyo 2 points3 points  (1 child)

    Do any software companies have full-time people contributing to the language? Have google or microsoft shown interest in Julia?

    [–]ViralBShah 9 points10 points  (0 children)

    Yes, various companies do so. Notably, Intel among the large companies. Among smaller firms, Invenia and Relational.ai contribute to Julia. We see a huge amount of interest from individual programmers at Google, Microsoft, and all the major firms. People already use Julia for small projects at these companies. With 1.0 out, we believe that there will be more production use cases as well.

    [–]trex005 2 points3 points  (3 children)

    It took quite a bit of work to see actual code samples while digging through your site. (admittedly on mobile)

    Generally with a new language, this is what I look for before all the "why it's so awesome"

    Is this just poor planning or are you afraid people will be turned off by the syntax and navigate away before learning the "why's"?

    [–]loladiro[S] 11 points12 points  (2 children)

    We've just changed up the design of the website and the sample code lost in the process. Issue about it here: https://github.com/JuliaLang/www.julialang.org/issues/115

    [–]trex005 5 points6 points  (1 child)

    I am comforted by the fact that this was noticed by a dev. I'll just chalk its absence to the fact that you were so busy over engineering a language that you under engineered the site.


    [–]loladiro[S] 5 points6 points  (0 children)

    Pull Requests to the website welcome ;)

    [–]alan_du 2 points3 points  (5 children)

    One of the trends (at least from my POV) in the data science and ML worlds is to more explicitly embrace the two-language problem by pushing more logic into the "lower-level" language (C/C++) while making the upper-level (relatively) thin bindings.. TensorFlow and Apache Arrow (both written in C++) are good examples of what I mean (especially compared to Scikit-Learn and Pandas respectively) -- most of their core logic is written in C++, but bindings are accessible to multiple languages (Java, C, Python, Go, JavaScript, etc), which allows different languages to interoperate more-or-less seamlessly (as long as you stick within the framework). Arguably Apache Spark's dataframe API is another example, where the core logic is Scala and it's accessible through Scala, Python, and R. BLAS/LAPACK might be the original example of this (although TBH I've only interacted with them through NumPy and SciPy).

    I guess I have two questions here:

    1. Do you think Julia is well-suited to writing some of the "infrastructure" libraries / frameworks? Is Julia well-suited to writing, e.g. a deep learning framework that's accessible through other languages or do you think C and C++ will continue to be the language(s) of choice here?
    2. What do you think of this trend in general? Do you think, for example, that Julia will end up binding to Apache Arrow for most of its dataframe heavy lifting, or do you think people will prefer to write their own versions in pure Julia?

    [–]loladiro[S] 7 points8 points  (2 children)

    Do you think Julia is well-suited to writing some of the "infrastructure" libraries / frameworks? Is Julia well-suited to writing, e.g. a deep learning framework that's accessible through other languages or do you think C and C++ will continue to be the language(s) of choice here?

    Yes, I think we'll see people doing more of this kind of thing with julia. There's already projects like https://github.com/JuliaDiffEq/diffeqr which expose Julia's differential equations ecosystem to R, and as more and more such killer use cases are developed in Julia, I fully expect people to want to call them from other languages.

    What do you think of this trend in general? Do you think, for example, that Julia will end up binding to Apache Arrow for most of its dataframe heavy lifting, or do you think people will prefer to write their own versions in pure Julia?

    The entire reason for Julia's existence is to be able to do everything in the same language. We have excellent support for bindings to other languages in order to re-use prior work, but I think if people use julia as yet another high level glue language only, we'll have failed. There's significant advantage to having everything in the same language, both to the user (being able to see how every part of the stack works and improve it for your use case) and the machine (doing analysis, e.g. automatic differentiation through all layers. You lose those when you go between languages.

    [–]alan_du 1 point2 points  (1 child)

    > Yes, I think we'll see people doing more of this kind of thing with julia. There's already projects like https://github.com/JuliaDiffEq/diffeqr which expose Julia's differential equations ecosystem to R, and as more and more such killer use cases are developed in Julia, I fully expect people to want to call them from other languages.

    That's super cool! I'm definitely going to take a look! Maybe this is me being selfish, but I would 100% recommend spending energy on exposing bindings to Julia libraries this as frictionless as possible. At least at $DAYJOB (and I imagine it's similar at a lot of places), we have a ton of investment in Python and C++ already and I can't see us moving away from that anytime soon, but if there are killer libraries in Julia exposed to Python we'd definitely want to use them!

    Out of curiosity -- how does the packaging and multiple dispatch work when exposing bindings to other languages? Does it only work with a handful of types, letting Julia pre-compile everything, or do you have to ship with the Julia language to JIT things?

    [–]loladiro[S] 1 point2 points  (0 children)

    Out of curiosity -- how does the packaging and multiple dispatch work when exposing bindings to other languages? Does it only work with a handful of types, letting Julia pre-compile everything, or do you have to ship with the Julia language to JIT things?

    Depends on what you want to do. These bindings tend to ship a full julia, as well as the capability to get a foreign language handle to julia objects (and sometimes more fancy things like direct memory sharing between julia and the host langauge). We've recently started replacing openlibm by julia implementations and there has been talk of offering those replacement implementations in a C-compatible libm library for others to use, which at that point would be a more limited form of binding that does not ship most of the julia runtime and would only ship the compute kernels themselves.

    [–]wallnuss 5 points6 points  (1 child)

    I would argue that Tensorflow is actually an anti-example, a lot of the important logic is implemented in Python and needs to be replicated by bindings in other languages.

    [–]alan_du 1 point2 points  (0 children)

    That's probably true (I suspect most of the training logic is Python-only for example), but we've had very few problems taking a SavedModel from a Python TensorFlow program and doing inference in C++ and Go (we did have to hack the TensorFlow build system a little bit to access certain ops, but everything seems to work).

    So I guess with TensorFlow it's more about the "core operations" in C++ with a ton of sugar in the respective languages -- which also makes some sense because every language has different idioms. I suspect that if Apache Arrow takes off, that's how things will work in Python -- I can't imagine abandoning Pandas anytime soon, but maybe the core memory interface and computations will happen through Arrow.

    TLDR: I agree I'm oversimplifying, but I think my point still stands. TensorFlow is a big framework, mostly written in C++, that I can deploy their models in a bunch of different languages (at least Python, C++, and Go. I don't know how well the other bindings work).

    [–]mustang__1 4 points5 points  (1 child)

    How will r/programmerhumor mock you?

    [–]loladiro[S] 12 points13 points  (0 children)

    Probably one based indexing, but I hope they can come up with something more clever.

    [–]MNCPA 6 points7 points  (2 children)

    Could you explain to the average person why another language is needed?

    [–]jeffbezanson 15 points16 points  (1 child)

    One would think, or maybe hope, that the language used to program computers would have been designed and decided on by great computer scientists decades ago, and from then on everybody just learns it. It was almost like that for a while, with languages like Lisp, Algol, and Pascal with solid academic roots that became kind of standards. But things took an interesting turn. People kept inventing new languages (e.g. python), and programmers often found those new languages greatly preferable to previous ones. The design space turned out to be much bigger than people thought, and there seems to be no end of work to do in figuring out new ways to balance and mediate the needs of computers and humans.

    [–]StefanKarpinski 14 points15 points  (0 children)

    This. The vast size of the programming language design space is hard to comprehend. I find that while people who use programming languages as tools often think that there can't possibly be that much more to explore here, people who work in programming language design invariably feel that we've barely scratched the surface and need to try a lot more things to learn what works and what doesn't.

    It's also not often appreciated just how much programming languages have changed and continue to change rapidly. "Goto considered harmful" was written not because people were overusing goto as compared to if/else and for loops—it was written to try to convince people to try these fancy new structured control flow constructs that had just been invented. And that was in 1968, only 50 years ago.

    In particular, most programming languages have special-cased numbers and arithmetic and baked these special things deep into the language spec. About 20% of C and Scheme language specs are devoted to spelling out how numbers and arithmetic work. Why? Why not just write code that implements them in terms of more basic functionality? Especially in Scheme which very much operates that way and tries to keep the core language as minimal as possible. The answer is because the language designs in question don't allow it—numbers and numeric operations are too polymorphic to be implemented in these languages. You can't define a function in C that behaves like the + operator or the [] indexing operators. In Python, you can kind of define a user + operation with __add__ and __radd__ but it's pretty awkward and it will be really slow. In Julia, Int is just a user-defined type that happens to be defined for you when you start the system and + is just a function like any other. Of course, as a user you don't care about that, but what you do care about is that your own types and operators are just as fast, efficient and powerful as the built-in ones. Julia programmers rely on this all the time and it's wildly powerful and lets them do things that are unthinkable in other languages.

    [–]cormullion 2 points3 points  (2 children)

    Where can I buy an official Julia t-shirt?

    [–]ViralBShah 8 points9 points  (1 child)

    A link is coming soon to julialang.org. And Cormullion we want some nice designs from you to feature there!

    [–]cormullion 6 points7 points  (0 children)

    Swag shop... :)

    [–]cormullion 2 points3 points  (5 children)

    What are the big changes you're scheduling for version 2.0? (Not too soon, I hope?)

    [–]StefanKarpinski 8 points9 points  (3 children)

    In terms of when, we're not sure, but a couple of years, so 2020-22. It also depends on how the technology for automatically upgrading code pans out. If it's trivial to upgrade a code base via FemtoCleaner 5.0 [1] then maybe Julia 2.0 will be a simple transition that we can do at any point.

    In terms of what, we're genuinely not sure—we need to regroup and take stock of things and spend a good long while on non-breaking improvements before we get into making new breaking changes.

    [1] https://github.com/JuliaComputing/FemtoCleaner.jl is an interactive bot that automatically upgrades Julia packages for you by making PRs on GitHub. It's currently mostly heuristic but it's pretty effective.

    [–]cormullion 1 point2 points  (0 children)

    It will nice to watch the dust settle for a while! :)

    [–]jeffbezanson 6 points7 points  (0 children)

    I'm hoping we can get some feature in place around interfaces/traits. It's felt like a big missing chunk of the language to many people. It's a tricky design problem though.

    There will probably be breaking changes to subtyping and specificity to make them as robust as possible. We also need to overhaul the macro hygiene system.

    [–]darf 2 points3 points  (2 children)

    I love the language. Could you share what you can about the business plan for Julia Computing? Also how you plan on evangelizing (other than being the best language, of course.)

    [–]loladiro[S] 6 points7 points  (0 children)

    Well, the business plan is fairly simple. Make enough money to pay people to make Julia amazing for everybody. The key is figuring out what we can charge for without compromising that goal. At JuliaCon, we announced that we're starting to do payed online training as well as a product called JuliaTeam, which should help companies that are using Julia manage and control which packages are available in their organization, as well as have a 1-click way to get support for various packages from us. There's a couple other things we're doing, but we hope that things like JuliaTeam will work well, both in terms of generating revenue, but also in giving companies confidence when deploying Julia in their organizations.

    [–]ViralBShah 9 points10 points  (0 children)

    The business plan for Julia Computing is to make products that make it easy for commercial users to adopt Julia, while at the same time, contributing to the open source project in a big way. Our focus for the immediate future is going to be training and JuliaTeam (see above).

    We do a variety of things to evangelize Julia. A large part of it is conferences, tutorials (on site and on Youtube Live), social media, and a new website! I must also mention that a grant from Sloan Foundation made it possible for us to have Jane Herriman as the Director of Diversity and Outreach, and we conducted a number of different activities and trainings as part of that effort. The best evangelization happens when people use Julia and show it off to their friends and colleagues, to their advisors, to their bosses, run local meetups, and so on.

    [–]t_bptm 2 points3 points  (2 children)

    It seems like programming has moved further away from mathematical to something... else. It seems like very few programmers actually know basic math (I mean basic from a mathematicians point of view, not that they can't add and subtract). What are your thoughts on this, and what are your thoughts on the effect this has had on the industry? Do you think Julia's popularity is a sign of this changing, that people want modern languages that allow them to bridge the gap between software and mathematics more easily?

    [–]loladiro[S] 5 points6 points  (0 children)

    I think knowing mathematics or rather the theory of whatever your working on is important to get things done and to make the right choices, but as you say, you can get far without it. What I think julia is very good at is to be able to describe concepts closely enough to the mathematical notation of the field that it is easier for programmers to make the jump from just trying to get things done to understanding the math. Take ML for example. A dense NN layer in Flux.jl is implemented like this: https://github.com/FluxML/Flux.jl/blob/master/src/layers/basic.jl#L78. Even if you don't know much about math, you can see that there's a matrix multiplication and an addition (which may not be so obvious if the same is implemented in a few thousand lines of C++ or CUDA). So I hope that Julia can help guide people to the underlying mathematical beauty.

    [–]jeffbezanson 3 points4 points  (0 children)

    Yes --- particularly with the level of interest in machine learning these days, numerical computing is back and bigger than ever. I've also always thought it would be nice for programming to be more like math in certain ways. Everybody learns some basic math in school, both because it's useful and to develop general thinking skills. I think programming is even better for that and should be just as universal. Math also understands the importance of notation. Math notation (while far from perfect at times) is an elegant and powerful tool honed over centuries, while programming notation is too often some quirky thing involving curly braces. We need to do better, and try to make programming truly worth learning for everybody.

    [–]qKrfKwMI 2 points3 points  (1 child)

    First of all, I'm a big fan of Julia ever since I tried to interface CUDA code with (numpy) python, where it took me a long time just to figure out what interfacing package I should use. The contrast with the ease of callling compiled functions from julia couldn't have been greater. I rewrote my python code to julia and I was up and running in about a week. By now I'm totally hooked and feel quite limited by other languages, either because of performance ("should I really be writing this python loop?") or missing multiple dispatch.

    • I get that you wanted to release during JuliaCon, but wasn't it perhaps better if you had more time between the releases of 0.7 and 1.0 so more packages could get themselves 1.0 compatible for the release? It felt a bit disappointing for me when I tried installing packages after the great 1.0 release and got errors.

    • The programming paradigm of Julia, with the focus on multiple dispatch, is quite different from other, standard languages. In my experience, I found it not obvious how I should approach writing a module and making a good, reusable interface, for instance the dilemma between "Let the user put a function in a struct and call the function as t.f(...) or let the user subtype and define f(t::MyType, ...)", I'm still not sure (when) which is the right choice. As a (non-computer) scientist first, my programming skills are limited, which probably adds to my confusion, but do you know if some resources/guidelines have been developed for this kind of thing?

    • How soon after the release did you discover the analogue of that typo you see right after you press send on an email: that one bad thing that could/should have been fixed but now you're stuck with it because of the stability guarantees of 1.0.

    Edit: Oh yeah, thank you so much for making this amazing language.

    [–]loladiro[S] 3 points4 points  (0 children)

    I get that you wanted to release during JuliaCon, but wasn't it perhaps better if you had more time between the releases of 0.7 and 1.0 so more packages could get themselves 1.0 compatible for the release? It felt a bit disappointing for me when I tried installing packages after the great 1.0 release and got errors.

    It's a trade off. It was a bit rushed perhaps, but all things considered I think it was the right decision. Doing it at JuliaCon allowed us to achieve this milestone as a community rather than just having a few core developers tag it and call it a day. Also, actually releasing the thing made the transition to 1.0 much faster. At this point we have about the same fraction of the ecosystem working on 1.0 as we did on 0.7 a week ago and things are still going up. The dust here will be settled in a week or two, but everyone's memory of the 1.0 tagging event will last much longer.

    The programming paradigm of Julia, with the focus on multiple dispatch, is quite different from other, standard languages. In my experience, I found it not obvious how I should approach writing a module and making a good, reusable interface, for instance the dilemma between "Let the user put a function in a struct and call the function as t.f(...)
    or let the user subtype and define f(t::MyType, ...)
    ", I'm still not sure (when) which is the right choice. As a (non-computer) scientist first, my programming skills are limited, which probably adds to my confusion, but do you know if some resources/guidelines have been developed for this kind of thing?

    It takes a bit of getting used to, but often some patterns will feel natural and some won't. The easiest way to get that intuition is to play with some of the more popular julia libraries (Distributions, ColorTypes, the Images.jl stack) and see what kind of patterns they use and in what situations they do. There are a couple of guides around, but I don't think they're a substitute for gaining that intuition.

    How soon after the release did you discover the analogue of that typo you see right after you press send on an email: that one bad thing that could/should have been fixed but now you're stuck with it because of the stability guarantees of 1.0.

    I don't think we've found that yet. I'm sure we will, but we did spend a long time getting 1.0 in shape to make sure that doesn't happen. There's always bugs of course, and there was one particular regression which would have been very embarrassing to ship 1.0 with (https://github.com/JuliaLang/julia/issues/28527), but luckily that was caught just before we were about to release and it made it into 1.0.

    [–]hobbies_only 2 points3 points  (4 children)

    I'm extremely interested in language design, I'd love to work on a language after school (masters CS student graduating in May). I built a compiler from scratch for a course which was beyond rewarding, and I've built a toy language with PLY (python lex/yacc).

    What tips do you have for someone interested in language design that would like to do it professionally? Should I be trying to contribute to an open source language? Should I continue to develop toy languages for experimentation?


    [–]loladiro[S] 1 point2 points  (3 children)

    Well, language design is a bit of a specialized field and there's not too many places doing it, particularly if you want to do actual language design yourself rather than just working on a language implementation. That said, there is a number of very active open source language communities out there. At Julia Computing, we tend to hire from people who have existing experience contributing to Julia in the open source community and I imagine companies supporting other languages work similarly. If you want to do toy languages, I'd recommend playing with a couple different kinds (imperative, functional, pattern matching, etc) and try out new techniques. And don't get too attached to any one of them or you may find your next decade taken up by actually implementing it fully ;).

    [–]hobbies_only 1 point2 points  (2 children)

    Great advice. Yes, because it seems a bit niche I'm finding it hard to learn more about. And I'll never want to let my baby languages go... but true nonetheless.

    Thank you!

    [–]fasquoika 2 points3 points  (1 child)

    Yes, because it seems a bit niche I'm finding it hard to learn more about.

    There's definitely some truth to this, but I've also found niche programming subjects to have a higher signal-to-noise ratio. When you find a resource about implementing programming languages, it tends to be written by someone who knows what they're talking about. I've recently been learning about how Guile Scheme is implemented, and reading Andy Wingo's blog posts on it (such as this one) have been illuminating. BTW you should take a look at r/ProgrammingLanguages, lots of interesting resources get posted there and the community is generally really helpful

    [–]hobbies_only 1 point2 points  (0 children)

    That's a great suggestion, thank you!

    And yes, you're absolutely right.

    [–]RandomizedAlex 2 points3 points  (2 children)

    Julia 1.0 is really a great release. What I miss the most compared to other languages is a debugger. Are there any plans for developing a debugger (and keeping it up-to-date with julia releases)?

    [–]loladiro[S] 4 points5 points  (1 child)

    I intend to update ASTInterpreter2 to 1.0, which is at least half of a debugger. You may also be interested in https://github.com/timholy/Rebugger.jl. I do have plans for a more fully featured, native, out-of-process debugger, but that's a bunch of work that I'll need to figure out how to find the time for.

    [–]deviluno 2 points3 points  (0 children)

    What's the expected time for easy static compilation out of the box? By that I mean, something like juliac -o prog program.jl just generates an optimized binary. I expect that many people want this.

    Same question for a debugger, as asked elsewhere.

    How about interfaces/signatures? Last time I checked this was a TODO. Will that be 2.0 or 1.X?

    switch/case statements? Any reason they're not in the language?

    OK, I have many such questions. Congratulations on the 1.0 release! Julia is a terrific language, and now that it is *finally* at 1.0 I'll try to use it in earnest. It's about time that numerical computing got a modern language.

    [–]yuvipanda 2 points3 points  (5 children)

    Thank you for your awesome work! <3 Julia!

    I'm curious how the VC funding affects the language and community in the long run. I (and others in the community) have had bad experiences with relying on foundational tech that is reliant on VC funding. When the VCs come to squeeze Julia Computing for returns, what happens to Julia?

    [–]kibretw 2 points3 points  (2 children)

    Why name it Julia, because Googling it will be a little hard?

    [–]star-castle 4 points5 points  (1 child)


    Julia's the first non-ad result, and I doubt ddg is as manipulative as Google. After Go and C doing as well as they are, you could probably name a language 'The'.

    [–]aexl 2 points3 points  (1 child)

    I have a math background and my primary language is rather Python, although I'm luring around the Julia community for many years now. I'm not that experienced in Julia, I've only converted my old numerical analysis exercises from MATLAB to Julia, but it was a lot of fun!

    Could you mention some beginner-friendly Julia packages, that are looking for contributors? I would love to give something back to the Julia community or maybe even get a part of it!

    [–]loladiro[S] 2 points3 points  (0 children)

    Depends on what you're looking for. We've been having a lot of fun getting students to reimplementing various research papers in machine learning (see https://github.com/FluxML/model-zoo/pulls) or differential equation solvers (various packages under https://github.com/JuliaDiffEq). Because of the 1.0 release, there's also various packages that still need some tweaks to work properly on 1.0, so if there's some package you find interesting, try installing 0.7 and see if you can't fix any warnings that come up (those warnings are generally errors in 1.0).

    [–]FoxesTrot 2 points3 points  (1 child)

    I don't really have a question, just wanted to say thanks!

    My undergraduate physics thesis was based on a project to speed up a nonlinear optics simulation done in matlab by 'translating' to Julia, it was great fun!

    I feel lame not asking a question in an AmA, so:

    What's your goto syntax colour scheme?

    [–]loladiro[S] 2 points3 points  (0 children)

    Thank you for stopping by. Glad we were able to be of help ;).

    My editors currently are all set to Monokai, but I'm not too picky on what the colors are. I do like having the colors though.

    [–]carzian 2 points3 points  (1 child)

    Julia looks like a fantastic language for data analysis, and I'm excited there's finally a good (free) alternative to Matlab. Recently it seems there's been a trend in the scientific community to switch to python over Matlab for increased flexibility when programming.

    I looked though the documentation and was slightly disappointed though. It seems there are a lot of small intricacies that would make this a more time consuming language to learn, similar to C++ or Ruby. Additionally, in the documentation it looked like I saw a lot of "you can totally do this bad practice coding, but you shouldn't because it's bad practice". I guess this is a more personal preference to like more strict programming languages but I think they're easier to work with while learning and in the long run. It looks like Julia is the opposite paradigm compared to languages like Go.

    For these reasons I think this will be fantastic for scientific computing, but I'm not seeing it as widely adopted for general purpose programming. I'm not trying to be negative because this looks like great work, I was just wondering to get your thoughts on this?

    [–]loladiro[S] 2 points3 points  (0 children)

    We tend to have a general philosophy of letting you do whatever you want in your program (I mean, hey it's your program after all - if there's an easy hack to get things done just do it) and then build tools to help you improve your code afterwards if you want to (if you're just building a one off, you might not care). Perhaps the most obvious of these tools at the moment are the various @code_typed-style introspection tools to tell you what the compiler was able to find out about your code and guide you to improving it for additional performance. There's a couple of extra ones that are less mature. I think we'll see more of that in the future. Julia is quite statically analyzable (which the compiler does for performance), so there's some very powerful tools that can be built for correctness, or paradigm adherence also. However, we really try to make sure that Julia "doesn't get in your way" if you're just trying to get something done and this dual approach seems to be a good way to get it done. They key is to make it integrated nicely to make sure that users actually take advantage of it. That'll be ongoing work, but I'm confident we'll be able to figure out a good way to do it.

    [–]NilacTheGrim 2 points3 points  (2 children)

    Programmer here. Never heard of Julia before. Took a quick look at your docs. This looks to me like Matlab m-file programming meets Golang.

    I guess this language was designed for academics familiar with Matlab to take the plunge and use a real compiled language for performance, etc?

    I know a number of scientists that are outgrowing Matlab for their needs. I guess Julia is designed to fill that niche?

    [–]loladiro[S] 1 point2 points  (1 child)

    It was designed for a similar user community. Matlab (and systems like it) got many things about the ergonomics scientific computing right, but many things about language design horribly wrong. The goal of Julia was to have something that was "non-scary" to scientists and domain experts, but have very strong foundations as a proper programming language. One of the most powerful things about julia is how composable and generic julia code tends to be (mostly thanks to the way that multiple dispatch works and meshes with specialization), which is not something you would find in either Matlab or Go. I'd encourage you to take a deeper look into the docs (and maybe even try the language). There's some very cool and powerful ideas in the language that you don't really find elsewhere.

    [–]NilacTheGrim 2 points3 points  (0 children)

    As someone who has worked in Matlab to develop integration components (Matlab APIs that call into my C++ libs) -- I agree with everything you just said about Matlab's strengths and weaknesses.

    In defense of the Matlab team -- their language is old and was developed somewhat hastily long ago. I suspect their rush to get to market and their focus on ease of use for the non-programmers led to some of their language choices or some of the lack of foresight. Also -- let's not forget the mid-90's were very different culturally from what he have now. The internet has opened everyone's minds up.

    /End Matlab defense

    I really like the work you're doing. The fact that it's compiled code and even the interpreter generates ASM is impressive. I read something briefly about dynamic code generation and other niceties. Pretty impressive if that's the case.

    I'd encourage you to take a deeper look into the docs (and maybe even try the language).

    I'm looking forward to it!

    I hope Julia takes root in scientific computing. I know all the scientists that learn to program on Matlab quickly outgrow it. Some learn C#, some move on to Python (not the best choice IMHO at least for numerical sims for various reasons), some resort back to Fortran or C++.

    Julia seems like it's the best of all worlds.

    Also -- from what I see you've avoided a lot of the weirdness about Go that some people love to complain about.

    Anyway I will look into it in more depth. Thanks for posting and great work so far!

    [–]kotipuka01 2 points3 points  (2 children)

    Hey, 17-year-old guy with very little programming experience here.
    So, I have a very basic knowledge of C++ from high school and I was wondering if Julia would be a good starter language to learn, since you mentioned that it is "dirt simple to learn"?
    If yes, would you point me to tutorials or books on Julia? :)

    [–]loladiro[S] 4 points5 points  (1 child)

    I think Julia in general is a good language to get started with, since it can span much of the abstraction stack in programming, from very low level bit manipulation to very high-level functional programming abstractions. For an introduction to Julia, you might like the recent JuliaCon introductory tutorial: https://www.youtube.com/watch?v=cPYgipsg4DM. Fair warning though, if you get started with Julia, you may be too frustrated by other languages to ever want to use something else again ;).

    [–]NakedPerson 3 points4 points  (6 children)

    Favorite movie?

    [–]ViralBShah 10 points11 points  (2 children)

    Dr. Strangelove or: How I Learned to Stop Worrying and Love the Bomb

    [–]StefanKarpinski 10 points11 points  (1 child)

    Woah, I didn't realize that was your favorite movie—that's also mine. Close second and third: Blade Runner (original, obviously) and Brazil. Clearly, I have a penchant for sci-fi.

    [–]loladiro[S] 4 points5 points  (0 children)

    Hard to choose, but I'd say some of my favorites are the Matrix, Good Will Hunting and Spaceballs. I am more of a Sci-Fi TV than a movie person though. I think I'd rank my favorite shows as Dark Matter, Stargate, Firefly, The Expanse, TNG, Voyager, DS9.

    [–]alanedelman 3 points4 points  (0 children)

    While tempted to say anything with JULIA roberts, that's really my wife.

    [–]Bdamkin54 3 points4 points  (2 children)

    I have two suggestions that would trigger a positive phase change in Julia's growth:

    1. Enable useful web assembly support for front and back end Julia programs.

    2. Convince a large company to use and work on Julia for their machine learning stack.

    Any plans to pursue these?

    [–]loladiro[S] 9 points10 points  (1 child)

    Enable useful web assembly support for front and back end Julia programs.

    Yes, this is something we're interested in, particularly because people are facing an analogue of the two language problem when they start trying to put Web UIs onto their Julia applications. There's a couple of approaches here (e.g. https://github.com/MikeInnes/Charlotte.jl, https://github.com/tshort/ExportWebAssembly.jl). I'm expecting that we'll see more of that in the near future.

    Convince a large company to use and work on Julia for their machine learning stack.

    We're talking to all the large companies but many of them have significant NIH syndrome until something gets super popular. That said, a few of them are seriously evaluating Julia, so hopefully something will come out of that.

    [–]Bdamkin54 1 point2 points  (0 children)

    Really glad to hear all that.

    And obviously #1 (along with multithreaded web server capability) will help with #2 .

    [–]cormullion 1 point2 points  (3 children)

    Jeff: classical or jazz violinist? Bach, Beethoven, Brahms, or Jean-Luc Ponty?

    [–]jeffbezanson 4 points5 points  (2 children)

    Classical. I play in http://cambridgesymphony.org/ . Not possible to pick among the *4* B's: Bach, Beethoven, Brahms, and Bartok! When I'm just playing at home in my spare time I often gravitate to the Bach solo suites though.

    [–]Johnfgibson 1 point2 points  (0 children)

    That's great. Beethoven & Bartok fanatic here. I've worked through much of Bartok Mikrokosmos on piano. Wagner, Stravinsky & Schoenberg are none too shabby, either.

    [–]PM_ME_A_PLANE_TICKET 1 point2 points  (1 child)

    Imagine I'm new to coding. If I wanted to expand my programming knowledge, should I learn Julia? What kinds of things would I be able to do? (web? mobile apps? something different?)

    [–]loladiro[S] 5 points6 points  (0 children)

    Julia excels at anything that involves heavy computation. These days, the hottest variant of that is probably machine learning. I did a few of the AI/ML MOOCs a few years ago and I think it would be quite pleasant to do the coding exercises for those in Julia. There is a number of julia tutorials available online if you're looking to get started.

    [–]Rojo424 1 point2 points  (1 child)

    Why's it called julia?

    [–]AtheistComic 1 point2 points  (1 child)

    What's an example of how Julia saves production time? What's an example of how you can lower overhead with Julia?

    [–]ViralBShah 5 points6 points  (0 children)

    The biggest saving in production comes from solving the two language problem. In many cases, you prototype in R or Python, and then someone rewrites in C++ or Java for production. Writing the same program twice is expensive and error prone, especially when the traditional production languages are quite verbose compared to the prototyping languages. With Julia, you can deploy what you develop, which is where the savings come from.

    [–]JMJimmy 1 point2 points  (4 children)

    Why is it that programming languages make architectural decisions that make them good/bad at doing certain tasks? Is it not possible to create one that simply has concurrent systems that can interact with each other to provide the most efficient architecture for a given task?

    [–]loladiro[S] 5 points6 points  (3 children)

    There's many trade offs in programming language design that can make it more suited for one task or another. For example, if your programming language is focused on shell scripting, you may not care about having operations for low level memory access, because your users don't tend to need that. Not implementing those operations make your job easier, and you can guarantee that the user won't shoot themselves in the foot by scribbling over the important memory. Another important consideration is implementation complexity. Some features are just very hard to implement. For example, Multiple Dispatch, which we do in Julia is great for expressing mathematical operations, but implementing it in a performant manner is really hard, and much of the design of Julia is geared towards making that possible. It would be basically impossible to take julia's multiple dispatch system and just bolt it onto an existing language in a high performance manner. That said though, money and time can make up for a lot of deficiency in language design. Modern JavaScript engines are true marvels of engineering and pretty damn fast, despite JavaScript not being a language that's very easy to optimize.

    [–]Vaglame 1 point2 points  (4 children)

    Any chance of someday see currying in Julia?

    [–]StefanKarpinski 4 points5 points  (1 child)

    Automatic currying is pretty fundamentally at odds with multiple dispatch since f(x) and f(x, y) are just different methods and there's no reason that f(x) should necessarily mean y -> f(x, y)—in fact, it's much more common for f(x) to call f(x, default) with some default value. So I don't think so. There is however, a pull request up for a concise partial evaluation syntax where f(x, _) would be syntax for y -> f(x, y). The main question with that syntax is how much of the surrounding code becomes the body of the anonymous function. That PR implements one possibility; I have a counter-proposal that I prefer that I'll post at some point soon once I've worked it out more thoroughly.

    [–]loladiro[S] 3 points4 points  (1 child)

    As an add-on to Stefan's reply, we do in general make comparison operators curried as of 1.0 to allow things like:

    findall(==(1), arr)

    which finds all elements of the collection `arr` equal (in the == sense) to 1.

    [–]Eulante 1 point2 points  (4 children)

    I'm a former Computer Science student currently enrolled in a Msc Data Science. Is there any chance to contribute to the language, considered my knowledge? I would like to find something to apply what I'm currently studying and Julia seems cool to start with.

    [–]loladiro[S] 4 points5 points  (0 children)

    I started contributing in high school, so I'd say your overqualified already ;). The best way to get started is to try Julia on a problem you care about, see what issues you find, report them and (if you're up for a challenge) fix them. When I was in school, I used julia for all my numerical problem sets and lab reports and some fun generic functionality came out of that (e.g. SIUnits.jl and improvements to plotting packages). The best thing to fix is something that frustrates you when trying to work with it.

    [–]jeffbezanson 2 points3 points  (1 child)

    Definitely. There are many ways to contribute; people sometimes even start with something as simple as fixing typos in the documentation, which is appreciated. (It's also not so simple, since learning basic git and making your first PR requires getting over a significant hurdle.) The julia standard library (and parts of the compiler) are written in julia, so just by learning the language you're already in a good position to start contributing. Check out the "help wanted" or "good first issue" labels on github.

    [–]qKrfKwMI 1 point2 points  (1 child)

    Ooh, I just thought of another question:

    On the one hand as language developers you hack away at language issues/problems, on the other hand there are people who use (features of) Julia for projects in ways you never would have imagined ahead of time. How did you manage to stay close to those people, such that you know what they want and need (which will not always be obvious, even to the users themselves)?

    [–]loladiro[S] 2 points3 points  (0 children)

    On the one hand as language developers you hack away at language issues/problems, on the other hand there are people who use (features of) Julia for projects in ways you never would have imagined ahead of time. How did you manage to stay close to those people, such that you know what they want and need (which will not always be obvious, even to the users themselves)?

    Some people complain loudly on GitHub or elsewhere when things don't work for them, so we find out that way. We also try to work closely with people who have particularly interesting or challenging problems to make sure Julia works well for them (and we learn what doesn't work that way). For example, I worked on the Celeste project last year (https://www.nextplatform.com/2017/11/28/julia-language-delivers-petascale-hpc-performance/). We're still not through the list of all the improvements we could make that would have made that project easier.

    Since creating the company, we also have commercial users now of course, and we tend to hear about their problems.

    [–]Dathiks 1 point2 points  (1 child)

    For starters, how do you go about creating a new language? What do you Do? Heck, how do you integrate it with current technology?

    And what are the advantages to your new languages? What are the disadvantages? what is it good for?

    [–]ice109 1 point2 points  (3 children)

    i can't tell if anyone is still here but i'm curious how the consulting/fintech side of the business is going? are you guys getting much traction with quant shops/prop firms? other kinds? you have blackrock on your testimonials page but anyone other than them? it'd be cool if you had a list (maybe tough because they wouldn't want divulge) of firms that had julia codebases so we could apply to them with our newfound julia skills :)

    [–][deleted]  (2 children)