I’ve never played with FORTRAN, but I’ve done some linear algebra with matlab. Matlab was interesting for the native handling if matrices. What makes FORTRAN so good at linear algebra?
Matlab’s syntax for matrices actually derives from Fortran. There’s a lot of flexibility in Fortran’s array features for
multidimensional arrays
arrays of indeterminate and flexible length
vectorized operations on arrays without explicitly writing loops.
Because Fortran does not have a pointer in the sense of C, the Fortran compiler is free to make several optimization that a C compiler can’t. Compiled Fortran is often faster than C code that does the same thing.
the main thing that makes fortran preferable to C is the way it handles arrays and vectors. due to different pointer semantics, they can be laid out more efficiently in memory, meaning less operations need to be done for a given calculation.
Interesting. Is this a fundamental limitation of C or is it just more preferable and easier to use FORTRAN when implementing it?
Meaning could the same performance be achieved in C but most optimized libraries are already written so why bother? Or basically C can’t achieve the memory optimization at all?
you can get the same performance by using the restrict keyword in C.
basically, C allows pointer aliasing while fortran does not, which means C programs need to be able to handle cases when a value is accessed from multiple locations. fortran does not, so a lot of accesses can be optimized into immediates, or unrolled without guards.
restrict is a pinky-promise to the compiler that no overlapping takes place, e.g. that a value will only be accessed from one place. it’s basically rust ownership semantics without enforcement.
Does one even have to actually write Python code, except for frontends? I’d assume you just load the model, weights and maybe training data into pytorch/tensorflow.
And then of course talking to these servers can be in any language that has a library for it or even just handles network requests, although Python is a nice choice. Possibly the process of training models is more heavy on the Python dependencies than inference is, haven’t actually done anything with that though.
To be fair, Python is just glue for code written in lower level languages when it comes to AI
A lot of it is c in a python raincoat
The underlining linear algebra routines are written in… FORTRAN.
I’ve never played with FORTRAN, but I’ve done some linear algebra with matlab. Matlab was interesting for the native handling if matrices. What makes FORTRAN so good at linear algebra?
Matlab’s syntax for matrices actually derives from Fortran. There’s a lot of flexibility in Fortran’s array features for
Because Fortran does not have a pointer in the sense of C, the Fortran compiler is free to make several optimization that a C compiler can’t. Compiled Fortran is often faster than C code that does the same thing.
the main thing that makes fortran preferable to C is the way it handles arrays and vectors. due to different pointer semantics, they can be laid out more efficiently in memory, meaning less operations need to be done for a given calculation.
Interesting. Is this a fundamental limitation of C or is it just more preferable and easier to use FORTRAN when implementing it?
Meaning could the same performance be achieved in C but most optimized libraries are already written so why bother? Or basically C can’t achieve the memory optimization at all?
you can get the same performance by using the
restrictkeyword in C.basically, C allows pointer aliasing while fortran does not, which means C programs need to be able to handle cases when a value is accessed from multiple locations. fortran does not, so a lot of accesses can be optimized into immediates, or unrolled without guards.
restrictis a pinky-promise to the compiler that no overlapping takes place, e.g. that a value will only be accessed from one place. it’s basically rust ownership semantics without enforcement.That reminds me, I had a ride share driver named Blas, and I had to giggle and tell them about it.
Which can be ASM in a C raincoat
Which can be ASMR depending on pronunciation and tone of voice.
Pwease mewge my pwull wequest, senpai UwU
Thank you I suppose?
Does one even have to actually write Python code, except for frontends? I’d assume you just load the model, weights and maybe training data into pytorch/tensorflow.
Doesn’t seem to be the case, some popular servers:
And then of course talking to these servers can be in any language that has a library for it or even just handles network requests, although Python is a nice choice. Possibly the process of training models is more heavy on the Python dependencies than inference is, haven’t actually done anything with that though.