modularml / mojo Goto Github PK
View Code? Open in Web Editor NEWThe Mojo Programming Language
Home Page: https://docs.modular.com/mojo/manual/
License: Other
The Mojo Programming Language
Home Page: https://docs.modular.com/mojo/manual/
License: Other
Hi,
Any chance you could rename this as it clashes with the Mojo framework that's been established for over 10 years now in Perl and more recently in Javascript as well?
Thanks,
Chase
The url to join the discord is discorg.gg
instead of discord.gg
Hi, last night I was playing around in the notebook and ran into a few interesting sharp edges.
The struct I had in mind was a D3 inspired "line builder", where it takes an x and y function, and some points, and returns the transformed points with that function.
The builder was defined as follows:
from Vector import DynamicVector
from DType import DType
alias Vec2 = SIMD[DType.f32, 2]
fn constant(x: F32) -> F32:
return x
struct Line:
var x: fn(F32) -> F32
var y: fn(F32) -> F32
fn __init__(self&):
self.x = constant
self.y = constant
fn set_x(self&, x: fn(F32) -> F32):
self.x = x
fn set_y(self&, y: fn(F32) -> F32):
self.x = y
fn evaluate(self, points: DynamicVector[Vec2]) -> DynamicVector[Vec2]:
var out = DynamicVector[Vec2]()
for i in range(points.__len__()):
let p = points[i]
let x = p[0]
let y = p[1]
out.push_back(Vec2(self.x(x),self.y(y)))
return out
The goal was to follow something like the builder idiom. For the following here is some book keeping:
# no lambdas for now so a custom x here
fn my_x(x: F32) -> F32:
return x*x
points.push_back(Vec2(0,0))
points.push_back(Vec2(1,0))
points.push_back(Vec2(2,2))
points.push_back(Vec2(3,1))
Originally I wanted to write
let e = Line().set_x(my_x).set_y(constant).evaluate(points)
However that errors with:
error: Expression [7]:23:25: invalid call to 'set_x': invalid use of mutating method on rvalue of type 'Line'
let e = Line().set_x(my_x).set_y(constant).evaluate(points)
~~~~~~~~~~~~^~~~~~
Expression [5]:13:5: function declared here
fn set_x(self&, x: fn(F32) -> F32):
^
So I wrote it out as:
var l = Line()
l.set_x(my_x)
l.set_y(constant)
let e = l.evaluate(points)
Here is the second sharp edge
(see #39 about the e.__len__()
)
for i in range(e.__len__()):
print(e[i])
outputs:
[0.000000, 0.000000]
[1.000000, 0.000000]
[2.000000, 2.000000]
[3.000000, 1.000000]
In other words, evaluate
is still using the original self.x
, not the current self.x
.
I implemented this fibonacci function in pure python for speed comparison:
def fib(a):
if a <= 0:
return 0
elif a == 1 or a == 2:
return 1
else:
return fib(a-1) + fib(a-2)
When executing with mojo, it gives this error:
error: Expression [1]:8:12: cannot load non-register passable type into SSA register (compiler bug, please report!)
elif a == 1 or a == 2:
^
When using mojo syntax it works fine:
fn fib(a:Int)->Int:
if a <= 0:
return 0
elif a == 1 or a == 2:
return 1
else:
return fib(a-1) + fib(a-2)
11:blkio:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8580d647_314a_460f_adf7_099adeeafcc3.slice/cri-containerd-82dcf24cd2c103ee32c91f57eb4ffbf57c08d192905d3b03ecf3565775f2eaae.scope
10:devices:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8580d647_314a_460f_adf7_099adeeafcc3.slice/cri-containerd-82dcf24cd2c103ee32c91f57eb4ffbf57c08d192905d3b03ecf3565775f2eaae.scope
9:cpuset:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8580d647_314a_460f_adf7_099adeeafcc3.slice/cri-containerd-82dcf24cd2c103ee32c91f57eb4ffbf57c08d192905d3b03ecf3565775f2eaae.scope
8:freezer:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8580d647_314a_460f_adf7_099adeeafcc3.slice/cri-containerd-82dcf24cd2c103ee32c91f57eb4ffbf57c08d192905d3b03ecf3565775f2eaae.scope
7:cpu,cpuacct:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8580d647_314a_460f_adf7_099adeeafcc3.slice/cri-containerd-82dcf24cd2c103ee32c91f57eb4ffbf57c08d192905d3b03ecf3565775f2eaae.scope
6:memory:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8580d647_314a_460f_adf7_099adeeafcc3.slice/cri-containerd-82dcf24cd2c103ee32c91f57eb4ffbf57c08d192905d3b03ecf3565775f2eaae.scope
5:hugetlb:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8580d647_314a_460f_adf7_099adeeafcc3.slice/cri-containerd-82dcf24cd2c103ee32c91f57eb4ffbf57c08d192905d3b03ecf3565775f2eaae.scope
4:pids:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8580d647_314a_460f_adf7_099adeeafcc3.slice/cri-containerd-82dcf24cd2c103ee32c91f57eb4ffbf57c08d192905d3b03ecf3565775f2eaae.scope
3:perf_event:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8580d647_314a_460f_adf7_099adeeafcc3.slice/cri-containerd-82dcf24cd2c103ee32c91f57eb4ffbf57c08d192905d3b03ecf3565775f2eaae.scope
2:net_cls,net_prio:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8580d647_314a_460f_adf7_099adeeafcc3.slice/cri-containerd-82dcf24cd2c103ee32c91f57eb4ffbf57c08d192905d3b03ecf3565775f2eaae.scope
1:name=systemd:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8580d647_314a_460f_adf7_099adeeafcc3.slice/cri-containerd-82dcf24cd2c103ee32c91f57eb4ffbf57c08d192905d3b03ecf3565775f2eaae.scope
0::/
Pressing the Blue "+" does not show the New Launcher window. To create a new notebook, you have to go to File->New->Notebook.
Calling Math.sin on SIMD[DType.f32, 64] throws an "error: Expression [7]:7:1: no viable expansions found", whereas Math.sin on SIMD[DType.f32, 32] returns the sine of the arguments.
sin(SIMD[DType.f32, 64].splat(0))
throws an error, but sin(SIMD[DType.f32, 32].splat(0))
works.
11:hugetlb:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4f9d95_3f64_4c2d_9065_61c1abf7a6bd.slice/cri-containerd-33caf9d762db3022d2f3fe5fdfdd12bbcf87af0830ed4bc6de8dde33c98df6ff.scope
10:cpuset:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4f9d95_3f64_4c2d_9065_61c1abf7a6bd.slice/cri-containerd-33caf9d762db3022d2f3fe5fdfdd12bbcf87af0830ed4bc6de8dde33c98df6ff.scope
9:devices:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4f9d95_3f64_4c2d_9065_61c1abf7a6bd.slice/cri-containerd-33caf9d762db3022d2f3fe5fdfdd12bbcf87af0830ed4bc6de8dde33c98df6ff.scope
8:memory:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4f9d95_3f64_4c2d_9065_61c1abf7a6bd.slice/cri-containerd-33caf9d762db3022d2f3fe5fdfdd12bbcf87af0830ed4bc6de8dde33c98df6ff.scope
7:freezer:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4f9d95_3f64_4c2d_9065_61c1abf7a6bd.slice/cri-containerd-33caf9d762db3022d2f3fe5fdfdd12bbcf87af0830ed4bc6de8dde33c98df6ff.scope
6:cpu,cpuacct:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4f9d95_3f64_4c2d_9065_61c1abf7a6bd.slice/cri-containerd-33caf9d762db3022d2f3fe5fdfdd12bbcf87af0830ed4bc6de8dde33c98df6ff.scope
5:perf_event:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4f9d95_3f64_4c2d_9065_61c1abf7a6bd.slice/cri-containerd-33caf9d762db3022d2f3fe5fdfdd12bbcf87af0830ed4bc6de8dde33c98df6ff.scope
4:net_cls,net_prio:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4f9d95_3f64_4c2d_9065_61c1abf7a6bd.slice/cri-containerd-33caf9d762db3022d2f3fe5fdfdd12bbcf87af0830ed4bc6de8dde33c98df6ff.scope
3:pids:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4f9d95_3f64_4c2d_9065_61c1abf7a6bd.slice/cri-containerd-33caf9d762db3022d2f3fe5fdfdd12bbcf87af0830ed4bc6de8dde33c98df6ff.scope
2:blkio:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4f9d95_3f64_4c2d_9065_61c1abf7a6bd.slice/cri-containerd-33caf9d762db3022d2f3fe5fdfdd12bbcf87af0830ed4bc6de8dde33c98df6ff.scope
1:name=systemd:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4f9d95_3f64_4c2d_9065_61c1abf7a6bd.slice/cri-containerd-33caf9d762db3022d2f3fe5fdfdd12bbcf87af0830ed4bc6de8dde33c98df6ff.scope
0::/
I was following the tutorial and ran this code.
struct MyPair:
var first: Int
var second: Int
# We use 'fn' instead of 'def' here - we'll explain that soon
fn __init__(self&, first: Int, second: Int):
self.first = first
self.second = second
fn __lt__(self, rhs: MyPair) -> Bool:
return self.first < rhs.first or
(self.first == rhs.first and
self.second < rhs.second)
bar = MyPair(1, 2)
Then, due to fat fingers I ran this.
bar.1
And now ... the notebook is freezing. Should that happen?
Run above code in Jupyter environment.
I can't run this in the current run, because the notebook is frozen. But on restart I get this:
%%python
import subprocess
subprocess.Popen(["cat", "/proc/self/cgroup"])
11:pids:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode933f7da_a239_4255_abac_ea303b5b73f6.slice/cri-containerd-6a19f982a7ef533860a2ec149b3abf36098576554868c33860bd4a82d582dde3.scope
10:hugetlb:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode933f7da_a239_4255_abac_ea303b5b73f6.slice/cri-containerd-6a19f982a7ef533860a2ec149b3abf36098576554868c33860bd4a82d582dde3.scope
9:devices:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode933f7da_a239_4255_abac_ea303b5b73f6.slice/cri-containerd-6a19f982a7ef533860a2ec149b3abf36098576554868c33860bd4a82d582dde3.scope
8:cpuset:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode933f7da_a239_4255_abac_ea303b5b73f6.slice/cri-containerd-6a19f982a7ef533860a2ec149b3abf36098576554868c33860bd4a82d582dde3.scope
7:blkio:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode933f7da_a239_4255_abac_ea303b5b73f6.slice/cri-containerd-6a19f982a7ef533860a2ec149b3abf36098576554868c33860bd4a82d582dde3.scope
6:freezer:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode933f7da_a239_4255_abac_ea303b5b73f6.slice/cri-containerd-6a19f982a7ef533860a2ec149b3abf36098576554868c33860bd4a82d582dde3.scope
5:memory:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode933f7da_a239_4255_abac_ea303b5b73f6.slice/cri-containerd-6a19f982a7ef533860a2ec149b3abf36098576554868c33860bd4a82d582dde3.scope
4:net_cls,net_prio:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode933f7da_a239_4255_abac_ea303b5b73f6.slice/cri-containerd-6a19f982a7ef533860a2ec149b3abf36098576554868c33860bd4a82d582dde3.scope
3:cpu,cpuacct:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode933f7da_a239_4255_abac_ea303b5b73f6.slice/cri-containerd-6a19f982a7ef533860a2ec149b3abf36098576554868c33860bd4a82d582dde3.scope
2:perf_event:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode933f7da_a239_4255_abac_ea303b5b73f6.slice/cri-containerd-6a19f982a7ef533860a2ec149b3abf36098576554868c33860bd4a82d582dde3.scope
1:name=systemd:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode933f7da_a239_4255_abac_ea303b5b73f6.slice/cri-containerd-6a19f982a7ef533860a2ec149b3abf36098576554868c33860bd4a82d582dde3.scope
0::/
Python set literal syntax {1,2,3}
is not parsed in Mojo🔥. (It tries to interpret it as dictionary and expects a colon)
When I enter the following in a cell:
struct MyGenericType[Type: AnyType]:
var value: Type
fn __init__(self&, v: Type):
self.value = v
All cells in my notebook stop evaluating. I would expect the cells to continue evaluating.
However, if I copy the cell from the HelloMojo notebook that defines a generic Array into a fresh notebook, the cells evaluate fine. I do not understand why the simple example fails.
struct MyGenericType[Type: AnyType]:
var value: Type
fn __init__(self&, v: Type):
self.value = v
11:freezer:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eded8d2_162f_4b93_aa8d_4d5278d1fdd8.slice/cri-containerd-72c619273dbff076e50143c56e8a537d3ae76e7c1b2d7375d02c6627d4eca2c0.scope
10:pids:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eded8d2_162f_4b93_aa8d_4d5278d1fdd8.slice/cri-containerd-72c619273dbff076e50143c56e8a537d3ae76e7c1b2d7375d02c6627d4eca2c0.scope
9:hugetlb:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eded8d2_162f_4b93_aa8d_4d5278d1fdd8.slice/cri-containerd-72c619273dbff076e50143c56e8a537d3ae76e7c1b2d7375d02c6627d4eca2c0.scope
8:net_cls,net_prio:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eded8d2_162f_4b93_aa8d_4d5278d1fdd8.slice/cri-containerd-72c619273dbff076e50143c56e8a537d3ae76e7c1b2d7375d02c6627d4eca2c0.scope
7:blkio:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eded8d2_162f_4b93_aa8d_4d5278d1fdd8.slice/cri-containerd-72c619273dbff076e50143c56e8a537d3ae76e7c1b2d7375d02c6627d4eca2c0.scope
6:cpu,cpuacct:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eded8d2_162f_4b93_aa8d_4d5278d1fdd8.slice/cri-containerd-72c619273dbff076e50143c56e8a537d3ae76e7c1b2d7375d02c6627d4eca2c0.scope
5:cpuset:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eded8d2_162f_4b93_aa8d_4d5278d1fdd8.slice/cri-containerd-72c619273dbff076e50143c56e8a537d3ae76e7c1b2d7375d02c6627d4eca2c0.scope
4:devices:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eded8d2_162f_4b93_aa8d_4d5278d1fdd8.slice/cri-containerd-72c619273dbff076e50143c56e8a537d3ae76e7c1b2d7375d02c6627d4eca2c0.scope
3:perf_event:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eded8d2_162f_4b93_aa8d_4d5278d1fdd8.slice/cri-containerd-72c619273dbff076e50143c56e8a537d3ae76e7c1b2d7375d02c6627d4eca2c0.scope
2:memory:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eded8d2_162f_4b93_aa8d_4d5278d1fdd8.slice/cri-containerd-72c619273dbff076e50143c56e8a537d3ae76e7c1b2d7375d02c6627d4eca2c0.scope
1:name=systemd:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eded8d2_162f_4b93_aa8d_4d5278d1fdd8.slice/cri-containerd-72c619273dbff076e50143c56e8a537d3ae76e7c1b2d7375d02c6627d4eca2c0.scope
In most APIs, the rank of a multidimensional array would be assumed to be equivalent to the shape.
I noticed that the invocation for NDBuffer
is NDBuffer[rank, shape, dtype]()
. It would be ergonomic to have a constructor that assumes rank from shape, so it could just be NDBuffer[DimList(2,3), DType.ui32]()
,
or even better NDBuffer[(2,3), DType.ui32]()
String
type in fn
allows any type to be used
Create a notebook on the playground with:
from String import String
fn add_strict(a: Int, b: Int) -> Int:
print("\nrunning through int impl")
return b + a
fn add_strict(a: String, b: String) -> String:
print("\nrunning through string impl")
return a + b
print(add_strict("should this work: ", 8))
let eight: Int = 8
print(add_strict("should this work: ", eight))
print(add_strict("should this work: ", True))
print(add_strict(5, 10))
print(add_strict(True, False))
Resulting in everything that doesn't match the (Int,Int) overload running through (String,String) and concatenating
running through string impl
should this work: 8
running through string impl
should this work: 8
running through string impl
should this work: True
running through int impl
15
running through string impl
TrueFalse
With a single (Int, Int) fn trying to use any type other than Int
fails to compile correctly, but having String
either by itself or overloaded will allow any type through, is this intentional?
A strict-mode version of lambda
that supports multiple statements.
Similar to how fn
is like a strict-mode version of def
, it seems like Mojo might benefit from having a strict-mode version of lambda
. It would have the same distinctions as fn
does from def
, except unlike fn
, it would support type inference for arguments and return types where possible.
Furthermore, it would be great if these strict-mode lambdas could support multiple statements. Although Python has been hesitant to add support for multi-statement lambdas in the past, other languages similar to Mojo, such as Swift, Rust, Julia, and Nim, have demonstrated the advantages of supporting them. And given the current prevalence of asynchronous and functional-style programming, this seems like an area where Mojo would benefit from taking inspiration from these other languages.
As for the syntax, maybe the fn
keyword could be reused here, but in an anonymous way without specifying a function name. Or maybe the lambda
keyword could still be used, but be made strict-mode by default and support multiple statements in a way that is compatible with its current syntax. Or maybe inspiration could be taken from the syntaxes used by Swift closures, Rust closures, or Nim anonymous procs.
There are probably nuances to this feature that I haven't considered yet, so I'd be interested to hear what others think.
There is a compiler error when using the TestSuite
on Int
values. Consider the following code snippet
from Testing import TestSuite
let x: Int = 42
let y: Int = 42
let ts = TestSuite("testing")
ts.assertEqual(x, y)
When run on the Mojo playground I get the following error:
error: Expression [31]:22:19: invalid call to 'assertEqual': callee expects 2 input parameters but 0 were provided
ts.assertEqual(x, y)
~~~~~~~~~~~~~~^~~~~~
/.modular/Kernels/mojo/Stdlib/Testing.mojo:81:5: function declared here
fn assertEqual[
^
The above code snippet also doesn't work for assertAlmostEqual
. Also, as notied in the discord the code snippet below does work:
let x: SI32 = 42
let y: SI32 = 42
let ts = TestSuite("testing")
print(ts.assertEqual(x, y))
I got the access to Mojo playground today, but when I logged in the playground and start my server, it says "Spawn failed
The latest attempt to start your server has failed. Would you like to retry starting it?". Clicking the "Relaunch Server" does not work.
https://docs.modular.com/mojo/why-mojo.html
I have made some modifications, including a few that have already been reported by Elliot Waite in a previous issue, that you may want to incorporate. As there is no versioning yet, I have converted the comparison results into markdown format for easier tracking.
Original Text | Modified Text |
---|---|
When we started Modular, we had no |
When we started Modular, we had no intention of building a new programming language. But as we were building our platform with the intent to unify the world's ML/AI infrastructure, we realized that programming across the entire stack was too complicated. Additionally, we were writing a lot of MLIR by hand and not having a good time. |
What we wanted was an innovative and scalable programming model that could target accelerators and other heterogeneous systems that are pervasive in machine learning. This meant a programming language with powerful compile-time metaprogramming, integration of adaptive compilation techniques, caching throughout the compilation flow, and other |
What we wanted was an innovative and scalable programming model that could target accelerators and other heterogeneous systems that are pervasive in machine learning. This meant a programming language with powerful compile-time metaprogramming, integration of adaptive compilation techniques, caching throughout the compilation flow, and other features that are not supported by existing languages. |
And although accelerators are important, one of the most prevalent and sometimes overlooked |
And although accelerators are important, one of the most prevalent and sometimes overlooked "accelerators" is the host CPU. Nowadays, CPUs have numerous tensor-core-like accelerator blocks and other AI acceleration units, but they also serve as the "fallback" for operations that specialized accelerators don't handle, such as data loading, pre-processing and post-processing, and integrations with foreign systems. Therefore, it was clear that we couldn't lift AI with an "accelerator language" that only worked with specific processors. |
Applied AI systems need to address all these |
Applied AI systems need to address all these issues, and we decided there was no reason it couldn't be done with just one language. Hence, Mojo was born. |
We decided that our mission for Mojo would include innovations in compiler internals and support for current and emerging accelerators, but we |
We decided that our mission for Mojo would include innovations in compiler internals and support for current and emerging accelerators, but we saw no need to innovate in language syntax or community. Thus, we chose to embrace the Python ecosystem because it is widely used, it is loved by the AI community and it is really nice! |
Mojo as a member of the Python family | Mojo as a member of the Python family |
The Mojo language has lofty |
The Mojo language has lofty goals. We want full compatibility with the Python ecosystem, predictable low-level performance and low-level control. We need the ability to deploy subsets of code to accelerators, and we don’t want ecosystem fragmentation. We hope that people find our work useful over time and don’t want something like the Python 2 => Python 3 migration to happen again. These are not small goals! |
Fortunately, while Mojo is a brand new code base, we aren’t really starting from scratch conceptually. Embracing Python massively simplifies our design efforts, because most of the syntax is already specified. We can instead focus |
Fortunately, while Mojo is a brand new code base, we aren’t really starting from scratch conceptually. Embracing Python massively simplifies our design efforts, because most of the syntax is already specified. We can instead focus on building the compilation model and designing specific systems programming features. We also benefit from the tremendous work done on other languages (e.g. Clang, Rust, Swift, Julia, Zig, Nim, etc.), and leverage the MLIR compiler ecosystem. We also benefit from experience with the Swift programming language, which migrated most of a massive Objective-C community to a new language. |
Further, we decided that the right long-term goal for Mojo is to provide a superset of Python (i.e. be compatible with existing programs) and to embrace the CPython immediately for long-tail ecosystem enablement. To a Python programmer, we expect and hope that Mojo will be immediately familiar, while also providing new tools for developing systems-level code that enable you to do things that Python falls back to C and C++ for. We aren’t trying to convince the world that |
Further, we decided that the right long-term goal for Mojo is to provide a superset of Python (i.e. be compatible with existing programs) and to embrace the CPython immediately for long-tail ecosystem enablement. To a Python programmer, we expect and hope that Mojo will be immediately familiar, while also providing new tools for developing systems-level code that enable you to do things that Python falls back to C and C++ for. We aren’t trying to convince the world that "static is good" or "dynamic is good" - our belief is that both are good when used for the right applications, and that the language should enable the programmer to make the call. |
How compatible is Mojo with Python really? | How compatible is Mojo with Python really? |
Mojo already supports many core features of |
Mojo already supports many core features of Python, including async/await, error handling and variadics. However, since it is still in its early stages and missing many features, it is not yet very compatible with Python. In fact, Mojo doesn't even support classes yet! |
That said, we have experience with two major but different compatibility journeys: the |
That said, we have experience with two major but different compatibility journeys: the "Clang" compiler is a C, C++, and Objective-C (and CUDA, OpenCL, etc.) that is part of LLVM. A major goal of Clang was to be a "compatible replacement" for GCC, MSVC and other existing compilers. It is hard to make a direct comparison, but the complexity of the Clang problem appears to be an order of magnitude bigger than implementing a compatible replacement for Python. The journey there gives us good confidence that we can do this right for the Python community. |
Another example is the Swift programming language, which embraced the Objective-C runtime and language ecosystem and progressively shifted millions of programmers (and huge amounts of code) incrementally over to a completely different programming language. With Swift, we learned lessons about how to be |
Another example is the Swift programming language, which embraced the Objective-C runtime and language ecosystem and progressively shifted millions of programmers (and huge amounts of code) incrementally over to a completely different programming language. With Swift, we learned lessons about how to be "run-time compatible" and cooperate with a legacy runtime. In the case of Python and Mojo, we expect Mojo to cooperate directly with the CPython runtime and have similar support for integrating with CPython classes and objects without having to compile the code itself. This will allow us to talk to a massive ecosystem of existing code, but provide a progressive migration approach where incremental work put into migration will yield incremental benefits. |
Overall, we believe that the north star of compatibility, continued vigilance on design, and incremental progress towards full compatibility will get us to where we need to be in time. | Overall, we believe that the north star of compatibility, continued vigilance on design, and incremental progress towards full compatibility will get us to where we need to be in time. |
Intentional differences from Python | Intentional differences from Python |
While compatibility and migratability are key to success, we also want Mojo to be a first class language on its |
While compatibility and migratability are key to success, we also want Mojo to be a first class language on its own and not be hobbled by not being able to introduce new keywords or add a few grammar productions. As such, our approach to compatibility is twofold: |
We utilize CPython to run all existing Python3 code |
We utilize CPython to run all existing Python3 code "out of the box" without modification and use its runtime, unmodified, for full compatibility with the entire ecosystem. Running code this way will get no benefit from Mojo, but the sheer existence and availability of this ecosystem will rapidly accelerate the bring-up of Mojo and leverage the fact that Python is really great for high level programming already. |
We will provide a mechanical migrator that provides very good compatibility for people who want to move Python code to Mojo. For example, Mojo provides a backtick feature that allows use of any keyword as an identifier, providing a trivial mechanical migration path for code that uses those keywords as identifiers or keyword arguments. Code that migrates to Mojo can then utilize the advanced systems programming features. | We will provide a mechanical migrator that provides very good compatibility for people who want to move Python code to Mojo. For example, Mojo provides a backtick feature that allows the use of any keyword as an identifier, providing a trivial mechanical migration path for code that uses those keywords as identifiers or keyword arguments. Code that migrates to Mojo can then utilize the advanced systems programming features. |
Together, this allows Mojo to integrate well in a mostly-CPython world, but allows Mojo programmers |
Together, this allows Mojo to integrate well in a mostly-CPython world, but allows Mojo programmers to progressively move code (a module or file at a time) to Mojo. This approach was used and proved by the Objective-C to Swift migration that Apple performed. Swift code is able to subclass and utilize Objective-C classes, and programmers were able to adopt Swift incrementally in their applications. Swift also supports building APIs that are useful for Objective-C programmers, and we expect Mojo to be a great way to implement APIs for CPython as well. |
It will take some time to build Mojo and the migration support, but we feel confident that this will allow us to focus our energies and avoid distractions. We also think the relationship with CPython can build from both directions - |
It will take some time to build Mojo and the migration support, but we feel confident that this will allow us to focus our energies and avoid distractions. We also think the relationship with CPython can build from both directions - wouldn't it be cool if the CPython team eventually reimplemented the interpreter in Mojo instead of C? 🔥 |
Detailed Motivation: | Detailed Motivation: |
Mojo |
Mojo was initially created with the goal of bringing an innovative programming model to accelerators and other heterogeneous systems that are pervasive in machine learning. However, one of the most important and prevalent "accelerators" is actually the host CPU. These CPUs are getting lots of tensor-core-like accelerator blocks and other dedicated AI acceleration units, but they also play a vital role as a "fallback" to support operations the accelerators don’t. This includes tasks like data loading, pre-processing and post-processing, and integrations with foreign systems written, for example, in C++. |
As such, it became clear that we |
As such, it became clear that we couldn't build a limited accelerator language that targets a narrow subset of the problem (e.g. just work for tensors). We needed to support the full gamut of general purpose programming. At the same time, we didn't see a need to innovate in syntax or community, and so we decided to embrace and complete the Python ecosystem. |
Why Python? | Why Python? |
Python is the dominant force in |
Python is the dominant force not only in the field of machine learning but also in countless other fields. It is easy to learn, known by important cohorts of programmers (e.g. data scientists), has an amazing community, has tons of valuable packages, and has a wide variety of good tooling. Python supports the development of beautiful and expressive APIs through its dynamic programming features, which led machine learning frameworks like TensorFlow and PyTorch to embrace Python as a frontend to their high-performance runtimes implemented in C++. |
For Modular today, Python is a non-negotiable part of our API surface stack - this is dictated by our customers. Given that everything else in our stack is negotiable, it stands to reason that we should start from a |
For Modular today, Python is a non-negotiable part of our API surface stack - this is dictated by our customers. Given that everything else in our stack is negotiable, it stands to reason that we should start from a "Python First" approach. |
More subjectively, we feel that Python is a beautiful |
More subjectively, we feel that Python is a beautiful language, designed with simple and composable abstractions, eschewing needless punctuation, which is redundant in practice when indentation is used. It is also built with powerful (dynamic) metaprogramming features that provide a runway to extend Python according to our needs for Modular. We hope that those in the Python ecosystem will see our new direction as a step forward, taking Python to the next level by completing it instead of competing with it. |
What’s wrong with Python? | What’s wrong with Python? |
Python has well known |
Python has well known problems, most obviously, poor low-level performance and CPython implementation decisions like the GIL. While many active projects are underway to improve these challenges, the issues brought by Python go deeper and particularly impact the AI field. Instead of talking about those technical limitations, we'll talk about the implications of these limitations here in 2023. |
Note that |
Note that every time we refer to Python in this section, we are referring to the CPython implementation. We will discuss other implementations shortly. |
The two-world problem | The two-world problem |
For a variety of reasons, Python |
For a variety of reasons, Python isn't suitable for systems programming. Fortunately, Python has amazing strengths as a glue layer, and low-level bindings to C and C++ allow building libraries in C, C++ and many other languages with better performance characteristics. This is what has enabled things like NumPy, TensorFlow, PyTorch and a vast number of other libraries in the ecosystem. |
Unfortunately, while this approach is an effective way to |
Unfortunately, while this approach is an effective way to build high performance Python libraries, it comes with a cost: building these hybrid libraries is very complicated. It requires a low-level understanding of the internals of CPython, knowledge of C/C++/… programming (undermining one of the original goals of using Python in the first place), makes it difficult to evolve large frameworks, and (in the case of ML) pushes the world towards "graph-based" programming models, which have worse fundamental usability than "eager mode" systems. TensorFlow was an exemplar of this, but much of the effort in PyTorch 2 is focused around discovering graphs to enable more aggressive compilation methods. |
Beyond the fundamental nature of the two-world problem in terms of system complexity, it makes everything else in the ecosystem more complicated. Debuggers generally can’t step across Python and C code, and those that can aren’t widely accepted. It is a pain for the package ecosystems to deal C/C++ code instead of a single world. Projects like PyTorch with significant C++ investments are intentionally trying to move more of their codebase to Python because they know it gains usability. | Beyond the fundamental nature of the two-world problem in terms of system complexity, it makes everything else in the ecosystem more complicated. Debuggers generally can’t step across Python and C code, and those that can aren’t widely accepted. It is a pain for the package ecosystems to deal with C/C++ code instead of a single world. Projects like PyTorch with significant C++ investments are intentionally trying to move more of their codebase to Python because they know it gains usability. |
The three-world and N-world problem | The three-world and N-world problem |
The two-world problem is commonly felt across the Python ecosystem, but things are even worse for developers of machine learning frameworks. AI is pervasively accelerated, and those accelerators use bespoke programming languages like CUDA. While CUDA is a relative of C++, it has its |
The two-world problem is commonly felt across the Python ecosystem, but things are even worse for developers of machine learning frameworks. AI is pervasively accelerated, and those accelerators use bespoke programming languages like CUDA. While CUDA is a relative of C++, it has its special problems and limitations and does not have consistent tools like debuggers or profilers. It is also effectively locked to a single hardware maker! |
The AI world has an incredible amount of innovation on the hardware front, and as a consequence, complexity is spiraling out of control. There are now many attempts to build limited programming systems for accelerators (OpenCL, Sycl, OneAPI, …). This complexity explosion is continuing to increase and none of these systems solve the fundamental fragmentation in tools and ecosystem that is hurting the industry so badly. | The AI world has an incredible amount of innovation on the hardware front, and as a consequence, complexity is spiraling out of control. There are now many attempts to build limited programming systems for accelerators (OpenCL, Sycl, OneAPI, …). This complexity explosion is continuing to increase and none of these systems solve the fundamental fragmentation in tools and the ecosystem that is hurting the industry so badly. |
Mobile and server deployment Another challenge for the Python ecosystem is one of deployment. There are many facets to this, including folks who want to carefully control dependencies, some folks prefer to be able to deploy hermetically compiled |
Mobile and server deployment Another challenge for the Python ecosystem is one of deployment. There are many facets to this, including folks who want to carefully control dependencies, some folks prefer to be able to deploy hermetically compiled "a.out" files, and multithreading and performance are also very important. These are areas where we would like to see the Python ecosystem take steps forward. |
Related work: |
Related work: Other approaches to improve Python |
There are |
There are many approaches to improve Python, including recent work to speed up Python and replace the GIL, languages that look like Python but are subsets of it, and embedded DSLs that integrate with Python but are not first class languages. While we cannot provide an exhaustive list of all the efforts, we can discuss some of the challenges in these areas and why they are not suitable for Modular's use. |
Improving CPython and JIT compiling Python | Improving CPython and JIT compiling Python |
Recently, |
Recently, significant effort has been put into improving CPython performance and other implementation issues, resulting in huge benefits for the community. This work is fantastic because it incrementally improves the current CPython implementation. Python 3.11 has delivered performance improvements of 10-60% over Python 3.10 through internal improvements, and Python 3.12 aims to go further with a trace optimizer. Many other projects are attempting to tame the GIL, and projects like PyPy (among many others) have used JIT compilation and tracing approaches to speed up Python. |
While these are great efforts, they are not helpful in getting a unified language onto an accelerator. Many accelerators these days only support very limited dynamic features, or do so with terrible performance. Furthermore, systems programmers don't just seek "performance" they also typically want a lot of "predictability and control" over how a computation happens. | |
While we are |
While we are fans of these approaches and feel they are valuable and exciting to the community, they unfortunately do not satisfy our needs. We are looking to eliminate the need to use C or C++ within Python libraries, seek the highest possible performance, and cannot accept dynamic features at all in some cases. Therefore, these approaches do not help. |
Python subsets and other Python-like languages | Python subsets and other Python-like languages |
There are many attempts to build a |
There are many attempts to build a "deployable" Python, such as TorchScript from the PyTorch project. These are useful because they often provide low-dependence deployment solutions and sometimes have high performance. Because they use Python-like syntax, they can be easier to learn than a novel language. |
On the other hand, these languages have not seen wide adoption - because they are a subset, they generally |
On the other hand, these languages have not seen wide adoption - because they are a subset, they generally do not interoperate with the Python ecosystem, do not have fantastic tooling (e.g. debuggers), and often unilaterally change inconvenient behavior in Python, which breaks compatibility and fragments the ecosystem. For example, many of these change the behavior of simple integers to wrap instead of producing Python-compatible math. |
The |
The challenge with these approaches is that they attempt to solve a weak point of Python, but are not as good at Python's strong points. At best, they can provide a new alternative to C and C++ - but without solving the dynamic use cases of Python, they cannot solve the "two world problem". This approach drives fragmentation, and incompatibility makes migration difficult to impossible - recall how challenging the Python 2 to Python 3 migration was. |
Embedded DSLs in Python | Embedded DSLs in Python |
Another common approach is to build an embedded DSL in Python, typically installed with a Python decorator. There are many examples of this, |
Another common approach is to build an embedded DSL in Python, typically installed with a Python decorator. There are many examples of this, such as the @tf.function decorator in TensorFlow, the @triton.jit in OpenAI's Triton programming model, etc. A major benefit of these systems is that they maintain compatibility with all of the Python ecosystem tooling, and integrate natively into Python logic, allowing an embedded mini language to co-exist with the strengths of Python for dynamic use cases. |
Unfortunately, the embedded mini-languages provided by these systems often have surprising limitations, |
Unfortunately, the embedded mini-languages provided by these systems often have surprising limitations, don't integrate well with debuggers and other workflow tooling, and do not support the level of native language integration that we seek for a language that unifies heterogeneous compute and is the primary way to write large scale kernels and systems. We hope to move the usability of the overall system forward by simplifying things and making it more consistent. Embedded DSLs are an expedient way to get demos up and running, but we are willing to put in the additional effort and work to provide better usability and predictability for our use case. |
Hello. Thank you for the awesome project. As a new project, I would suggest that Mojo breaks with Python on some syntactical points where Python only maintains undesirable behavior for legacy reasons.
Several aspects of Python syntax are maintained only because changing them would be too disruptive. Examples are as follows:
bool
evaluate to 1 for arithmetic operationsThe design choices for the Starlark language, which also aims to be similar to Python, may be highly informative.
Although this will introduce conversion overhead, it may also reveal many hidden bugs in current Python software.
https://github.com/bazelbuild/starlark/blob/master/design.md
Many aspects of Python are maintained for legacy purposes only, as (former) BDFL has also stated. As a new language which requires some manual fixes during conversion in any case, I think that fixing those legacy issues may be helpful.
The basic constructor should accept a DTypePointer instead of a bare MLIR pointer
I think it would be a nice feature to have a 2 letter file extension supported, such as .mj, this would be supported of course alongside .mojo and 🔥 and does not need to replace either.
I think this would be a good new file extension that supersedes the 2 letter file extensions people are used to typing and seeing with python such as py, it's also very efficient to type each, for example on a traditional keyboard, both the P and the Y keys are close to each other and both can be hit with the right hand quickly in succession.
mj are 2 letters that are also in close proximity and both on the right half of the traditional keyboard, making it quick and efficient to type, just as much or more than py is.
It also follows the theme of removing the vowels to create shorter file extensions, for example:
Ruby is supported as it's vowel-less version: rb
Other popular 2 character file extensions for programming languages:
Typescript: ts
Javascript: js
and even a history of removing vowels for shortening, for non-programming language file types such as:
Jpeg gets turned into: jpg
Mpeg gets turned into mpg
Overall for both readability and practicality sake, if you were to make a file named main.mojo, then effectively half the file name you have to read or type out ends up being the extension itself, and sure there is another option which is 🔥, but I think on most desktops that is even more difficult and impractical to type out, since you need to open an emoji window or hotkey etc afaik
TBD
Add to Mojo, support for Python backend development frameworks like Django and FastAPI
The same reason people use Python for backend development! You don't need to learn another programming language to be able to put your ideas on the internet
Be able to create RestAPIs, manage relational databases...
String variables declared with let
are mutable.
It's not clear whether the behavior below is intended or not. However, it's surprising that the code below runs since variables declared with let
are in principle supposed to be immutable.
from String import String
let x: String
x = "This is x's original value"
print(x)
x = "This is x's new value"
print(x)
let z: String = "This is z's value"
x = z
print(x)
Output:
This is x's original value
This is x's new value
This is z's value
They aren't included, which means users don't know what fields they can access.
Getting this error when I think the compiler should be able to do type inference for an expression:
error: Expression [9]:10:22: true value of type 'FloatLiteral' is not compatible with false value 'SIMD[f32, 1]' in conditional
let z: F32 = 1.0 if x != 0 else foo()
Following along in the playground, I tried this modification (more Pythonic)
def your_function():
let x: Int = 42
let y: F64 = 17.0
let z: F32 = 1.0 if x != 0 else foo()
print(z)
def foo() -> F32:
return 3.14
your_function()
If I change to
let z: F32 = F32(1.0) if x != 0 else foo()
to explicitly cast 1.0 as F32 then it works.
z
to be a python-style ternary assignment1.0
literal value, with the other branch returning an F32 from a function.this gives:
11:blkio:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5289b235_2fe6_491d_a283_04bf300492a0.slice/cri-containerd-614f157018766ac78b42ff32df864b1aed0b15f527eeab85e2e8198c6d46500c.scope
10:devices:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5289b235_2fe6_491d_a283_04bf300492a0.slice/cri-containerd-614f157018766ac78b42ff32df864b1aed0b15f527eeab85e2e8198c6d46500c.scope
9:cpuset:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5289b235_2fe6_491d_a283_04bf300492a0.slice/cri-containerd-614f157018766ac78b42ff32df864b1aed0b15f527eeab85e2e8198c6d46500c.scope
8:freezer:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5289b235_2fe6_491d_a283_04bf300492a0.slice/cri-containerd-614f157018766ac78b42ff32df864b1aed0b15f527eeab85e2e8198c6d46500c.scope
7:cpu,cpuacct:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5289b235_2fe6_491d_a283_04bf300492a0.slice/cri-containerd-614f157018766ac78b42ff32df864b1aed0b15f527eeab85e2e8198c6d46500c.scope
6:memory:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5289b235_2fe6_491d_a283_04bf300492a0.slice/cri-containerd-614f157018766ac78b42ff32df864b1aed0b15f527eeab85e2e8198c6d46500c.scope
5:hugetlb:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5289b235_2fe6_491d_a283_04bf300492a0.slice/cri-containerd-614f157018766ac78b42ff32df864b1aed0b15f527eeab85e2e8198c6d46500c.scope
4:pids:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5289b235_2fe6_491d_a283_04bf300492a0.slice/cri-containerd-614f157018766ac78b42ff32df864b1aed0b15f527eeab85e2e8198c6d46500c.scope
3:perf_event:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5289b235_2fe6_491d_a283_04bf300492a0.slice/cri-containerd-614f157018766ac78b42ff32df864b1aed0b15f527eeab85e2e8198c6d46500c.scope
2:net_cls,net_prio:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5289b235_2fe6_491d_a283_04bf300492a0.slice/cri-containerd-614f157018766ac78b42ff32df864b1aed0b15f527eeab85e2e8198c6d46500c.scope
1:name=systemd:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5289b235_2fe6_491d_a283_04bf300492a0.slice/cri-containerd-614f157018766ac78b42ff32df864b1aed0b15f527eeab85e2e8198c6d46500c.scope
0::/
The example code is:
struct MyString:
var data: Pointer[Int8]
# StringRef has a data + length field
def __init__(self&, input: StringRef):
let data = Pointer[Int8].alloc(input.length+1)
data.memcpy(first.data, input.length)
data[input.length] = 0
self.data = Pointer[Int8](data)
def __del__(owned self):
self.data.free()
The following line is wrong:
data.memcpy(first.data, input.length)
should have been
data.memcpy(input.data, input.length)
Currently it is not possible to assign to an item/subscript of a Python object (such as a list index or a dictionary entry) from Mojo code using the typical Python syntax of object[key] = value
. Attempting to do so results in an error indicating that the expression is not mutable, even if the dictionary is in a variable defined using var
:
from PythonInterface import Python
from PythonObject import PythonObject
let python_builtins: PythonObject = Python.import_module("builtins")
let dict: PythonObject = python_builtins.dict;
var my_dict: PythonObject = dict()
my_dict["x"] = "hello, world!";
error: Expression [1]:24:12: expression must be mutable in assignment
my_dict["x"] = "hello, world!";
~~~~~~~^~~~~
This same error occurs for any Python value (represented by the PythonObject type in Mojo), such as for lists:
error: Expression [4]:34:12: expression must be mutable in assignment
my_list[0] = "bonjour!"
~~~~~~~^~~
Many Python APIs take dictionaries as arguments, either as collections of data, or as parameters/options, so it's valuable to be able to construct these values from Mojo. This is already possible by manually calling the __setitem__
method that Python uses to implement item assignment, but this is a bit clunky and not what Python programmers are used to.
my_dict.__setitem__("x", "hello, world")
my_list.__setitem__(0, "bonjour!")
It would be preferable to be able to populate dictionaries using the typical syntax instead, and generally to use the same syntax as Python for such operations on any Python object.
It should be possible to assign to any Python objects that support __setitem__
(including but not limited to dictionary keys and list indices) using the typical Python syntax:
my_dict["x"] = "hello, world!"
my_list[0] = "bonjour!"
This was originally mentioned at https://discord.com/channels/1087530497313357884/1103478447872950312.
(Note that getitem support already appears to be implemented, I'm able to
print(my_dict["x"])
and print(my_list[0])
without a similar error.)
I defined:
fn fact(n : Int) -> Int:
fn factint(p : Int, m : Int) -> Int:
if m == 0:
return p
return factint(p*m, m-1)
return factint(1, n)
and got the compilation error:
error: Expression [6]:5:1: cyclic reference between expressions defining and using parameters
fn fact(n : Int) -> Int:
^
Expression [6]:6:5: parameter "factint($Int::Int,$Int::Int)" is defined here, which references itself
fn factint(p : Int, m : Int) -> Int:
^
A simpler recursive fact
without the tail-recursive helper does compile, and a top-level factint
compiles, so the problem seems to be related to factint
being internal to fact
.
11:blkio:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeaac2d5_a246_41df_9300_b56906cac38e.slice/cri-containerd-c1fa9d6cdf96a39b653a771aca2c6190bba02cf664ccb2525e5543afa25f80c2.scope
10:devices:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeaac2d5_a246_41df_9300_b56906cac38e.slice/cri-containerd-c1fa9d6cdf96a39b653a771aca2c6190bba02cf664ccb2525e5543afa25f80c2.scope
9:cpuset:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeaac2d5_a246_41df_9300_b56906cac38e.slice/cri-containerd-c1fa9d6cdf96a39b653a771aca2c6190bba02cf664ccb2525e5543afa25f80c2.scope
8:freezer:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeaac2d5_a246_41df_9300_b56906cac38e.slice/cri-containerd-c1fa9d6cdf96a39b653a771aca2c6190bba02cf664ccb2525e5543afa25f80c2.scope
7:cpu,cpuacct:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeaac2d5_a246_41df_9300_b56906cac38e.slice/cri-containerd-c1fa9d6cdf96a39b653a771aca2c6190bba02cf664ccb2525e5543afa25f80c2.scope
6:memory:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeaac2d5_a246_41df_9300_b56906cac38e.slice/cri-containerd-c1fa9d6cdf96a39b653a771aca2c6190bba02cf664ccb2525e5543afa25f80c2.scope
5:hugetlb:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeaac2d5_a246_41df_9300_b56906cac38e.slice/cri-containerd-c1fa9d6cdf96a39b653a771aca2c6190bba02cf664ccb2525e5543afa25f80c2.scope
4:pids:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeaac2d5_a246_41df_9300_b56906cac38e.slice/cri-containerd-c1fa9d6cdf96a39b653a771aca2c6190bba02cf664ccb2525e5543afa25f80c2.scope
3:perf_event:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeaac2d5_a246_41df_9300_b56906cac38e.slice/cri-containerd-c1fa9d6cdf96a39b653a771aca2c6190bba02cf664ccb2525e5543afa25f80c2.scope
2:net_cls,net_prio:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeaac2d5_a246_41df_9300_b56906cac38e.slice/cri-containerd-c1fa9d6cdf96a39b653a771aca2c6190bba02cf664ccb2525e5543afa25f80c2.scope
1:name=systemd:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeaac2d5_a246_41df_9300_b56906cac38e.slice/cri-containerd-c1fa9d6cdf96a39b653a771aca2c6190bba02cf664ccb2525e5543afa25f80c2.scope
0::/
The ability to use mojo in Google Colab once Mojo reaches general availability.
Many ML enthusiasts use Google Colab to train their models. They can benefit from the code speedups that come with using Mojo.
Users should be able to download Mojo in Colab and use it without issue.
I propose to consider adding TypeScript-style union types to Mojo — instead of Rust-style nominal enums. This approach seems like it would work well given that Mojo plans to extend Python, which is a dynamically-typed language. In addition to enabling type declarations for Python libraries, union types would allow Mojo users to avoid the modelling issues associated with nominal enums.
Union types are strictly more general than nominal enums, and so have the potential to make enums obsolete.
Update Jan 2024: I removed a section that was too speculative, and I posted new thoughts on how enums/unions might be modelled using dependent types. This approach would supercede most of the design details discussed in this original post.
Most statically-typed languages that implement type-safe enums use nominal typing. For example, in Rust, to define a function that can return two kinds of error, one must first define (and name) an enumeration of those error kinds:
enum IOParseError {
IOError(...),
ParseError(...),
}
One can then specify the return type of the error-producing function as follows:
fn parseFile(file: &str) -> Result<Success, IOParseError>
This approach doesn't scale when different functions can return different combinations of errors. For example, maybe we have:
parseFile
(above) that produces either an IOError
or a ParseError
.readFile
that only produces an IOError
.parseString
that only produces a ParseError
.parseAndExecuteFile
that produces either an IOError
, a ParseError
, or a RuntimeError
.The issue here is that each combination of errors requires a different enum to be declared, and each enum needs to have unique constructor names. Ultimately, this makes expressing these return types using nominal enums a nightmare. (And this problem isn't unique to error types. There are many other scenarios in which this issue occurs.)
In contrast, modelling these functions in a dynamically-typed language (such as Python) is trivial. For example, the implementation of parseFile
will simply return whatever error object is appropriate. No up-front declaration is required.
Because of this flexibility afforded by dynamically-typed languages, retroactively adding a type system to such languages has proved challenging. But in recent years, TypeScript has demonstrated that it is possible to statically-type these kinds of programs. TypeScript's trick is to offer union types, via a union operator |
. The union operator can be used to directly express a disjunction of types. We can use the union operator to directly enumerate the specific errors that the aforementioned functions can return:
function parseFile(): Result<Success, IOError | ParseError> {...}
function readFile(): Result<Success, IOError> {...}
function parseString(): Result<Success, ParseError> {...}
function parseAndExecuteFile(): Result<Success, IOError | ParseError | RuntimeError> {...}
For brevity, I will omit a full description of how TypeScript's union types work. For more information, check out the relevant section of the TypeScript handbook.
Given that Python is a dynamically-typed language, Python programmers mix values of different types together in myriad ways. For example, the use of heterogeneous collections is common amongst Python programmers.
Accordingly, it would be beneficial if Mojo's static semantics could have a richer understanding of Python, so that more Python code is able to "just work" when copy-pasted into Mojo — even if the code is pasted into a statically-checked fn
.
Notably, Python already supports the use of union types as part of its type annotations feature. However, Python assigns no static semantics to these annotations. In this post, I'm arguing that giving union types a rigorous static semantics will enable Mojo users to construct expressive, statically-typed code that can be compiled very efficiently.
Developing a precise semantics for union types will require a large amount of work. Thankfully, TypeScript is a helpful piece of prior art. Scala 3 has union types as well. But as I explain below, I think we would want to differ in the fine details.
I'm unable to dedicate the time required to flesh out a full design proposal. However, I can skim over some potential questions and challenges that may arise:
Foo
and Bar
have a common method (not a trait method), should a person with access to a value of type Foo | Bar
be able to call it? (TypeScript allows this.)
Foo
or Bar
and calls a method that they have in common, then you should put that method in a protocol P
, have Foo
and Bar
implement that protocol, and then have your function accept an instance of P
rather than Foo | Bar
.Foo | Bar
, because you want to be able to pattern match on the argument and handle each struct differently. If Foo
and Bar
implement a common protocol P
, should the methods of that protocol be callable? This time I think the answer is yes, this should be allowed. Indeed, I relied upon this behaviour earlier, when I described how union types could obviate "trait objects". In short: it should always be possible to upcast a value of type Foo | Bar
to a value of type P
.Foo
and Bar
were protocols (rather than structs). A common method should only be callable on Foo | Bar
if the method belongs to a protocol that both Foo
and Bar
inherit from. (This is assuming Mojo supports protocol inheritance.)type Color = Red | Green | Blue
(where Red
, Green
, and Blue
are pre-defined struct types with no fields), there is no reason that the compiler can't encode them via the integers 0
, 1
and 2
when storing them in variables of type Color
and when passing them as arguments of type Color
. A difficulty arises when the same structs are used in multiple enums, e.g. type TrafficLight = Red | Yellow | Green
. When these two types "interact" (e.g. a Color
value is used as a TrafficLight
argument), re-mapping of the tag values may need to occur, e.g. Green
gets mapped from 1
to 2
. Alternatively, the compiler can avoid the need for such re-mappings by monomorphizing function calls. For example, given a function with an argument of type TrafficLight
, the compiler could generate a "special" version of the function where the tag of each TrafficLight
color is expected to match the tag for the respective Color
color (i.e. Green
is encoded as 1
). Call sites that provide a Color
as an argument will invoke this version of the function.__init__
methods are not invertible. Thus, we would probably need a keyword or decorator to designate that a struct is destructurable. The @value
decorator that Mojo offers could be used for this. In fact, Python's @dataclass
decorator already works this way for classes. A more general approach would be to allow structs to define their own destructuring behaviour — analogous to how __getitem__
works, but for destructuring rather than indexing.if x is Foo(a, b, c)...
(a hypothetical syntax for destructuring x
), one would write x is Foo
, which merely tests whether x
is an instance of the struct Foo
. This test would be implemented in the same efficient manner as enums, as discussed earlier.None | None = None
. How should unification be treated?
Optional[T]
as follows:type Optional[T] = T | None
Optional[None]
is just None
, and that Optional[Optional[Int]]
is just Optional[Int]
. This seems hard to reason about!type Optional[T] = Some[T] | None
instead (where Some
is a pre-defined struct). It seems reasonable to impose this restriction, given that it remains strictly more expressive than traditional enums.
None | None
doesn't seem so nefarious — and such unification will obviously happen during type inference.This is probably enough detail for the moment. What does the Mojo team think of this approach?
I noticed that the current syntax for declaring functions is fn
as well as def
, but I believe it would be helpful to provide an alternative syntax using the keyword function.-->
I would like to request that you consider adding the function
keyword as an alternative way to define functions, in addition to the existing fn
and def
syntax.
What is the value to the product/user of doing this?
This would allow developers to choose the syntax that they are most comfortable with, and would also make the language more accessible to those who are used to other programming languages that use the function
keyword. Per se I'm a swift programmer.
I understand that this may require some changes to the codebase, but I believe it would ultimately make the language more flexible and easier to use for a wider range of developers.
Thank you for your consideration, and please let me know if there is any way I can help adding this feature or any concerns about this request.
Support for model inference in web browsers and web-compatible server-side JS runtimes like Deno.
There has been an explosion in recent months of various ML projects/demos/libraries on the web. These were all launched within the last couple of months:
This is in part thanks to the launch of WebGPU, and partly thanks to the years of toil from contributors in projects such as ONNX Runtime Web.
I expect the ecosystem to grow quite quickly in the coming months and years, especially as the planned ML-specific extensions are added to the WebGPU standard.
The Modular keynote mentioned edge deployment, so I'm hoping web is already included in the planned roadmap, but figured I'd make an issue just in case.
What’s the motivation behind the choice of the self&
syntax? Coming from Rust and C++, the &self
syntax seems more natural to me. And having the &
be a prefix seems like it would align more with how borrowed
and owned
are placed before the argument name.
The program below crashes with a double-free error.
Run the following program in a notebook:
from String import String
struct Foo:
var x: String
fn __init__(self&, owned x: String):
self.x = x
fn __moveinit__(self&, owned existing: Self):
self.x = existing.x
fn bar(owned foo: Foo) -> Int:
return 2
let foo = Foo("test")
print(bar(foo^))
print(bar(foo^))
Reported by @dom96 on Discord.
String literals do not persist across cells:
Cell 1: var x = "hello"
.
Cell 2: print(x)
Expected result: "hello" printed
Actual result: Blank line printed
Concatenation with a string literal stored in a variable causes an error
var x = "Hello"
print(x)
print("hi" + x)
Expected result: "hihello" printed
Actual result: Error: failed to legalize operation 'pop.string.concat' that was explicitly marked illegal
Simple reproduction:
alias Vec2 = SIMD[DType.f32, 2]
var points = DynamicVector[Vec2]()
points.push_back(Vec2(0,0))
points.push_back(Vec2(1,0))
points.push_back(Vec2(2,2))
points.push_back(Vec2(3,1))
print(len(points))
output:
error: Expression [10]:23:14: no matching function in call to 'len':
print(len(points))
~~~^~~~~~~~
/.modular/Kernels/mojo/Stdlib/Len.mojo:20:1: candidate not viable: argument #0 cannot be converted from 'DynamicVector[SIMD[f32, 2]]' to 'String'
fn len(value: String) -> Int:
^
/.modular/Kernels/mojo/Stdlib/Len.mojo:32:1: candidate not viable: argument #0 cannot be converted from 'DynamicVector[SIMD[f32, 2]]' to 'StringRef'
fn len(value: StringRef) -> Int:
^
/.modular/Kernels/mojo/Stdlib/Len.mojo:44:1: candidate not viable: argument #0 cannot be converted from 'DynamicVector[SIMD[f32, 2]]' to 'StringLiteral'
fn len(value: StringLiteral) -> Int:
^
/.modular/Kernels/mojo/Stdlib/Len.mojo:56:1: candidate not viable: callee expects 2 input parameters but 0 were provided
fn len[size: Dim, type: DType](value: Buffer[size, type]) -> Int:
^
/.modular/Kernels/mojo/Stdlib/Len.mojo:68:1: candidate not viable: callee expects 1 input parameter but 0 were provided
fn len[type: AnyType](value: VariadicList[type]) -> Int:
^
/.modular/Kernels/mojo/Stdlib/Len.mojo:80:1: candidate not viable: callee expects 1 input parameter but 0 were provided
fn len[type: AnyType](value: VariadicListMem[type]) -> Int:
^
/.modular/Kernels/mojo/Stdlib/Len.mojo:93:1: candidate not viable: argument #0 cannot be converted from 'DynamicVector[SIMD[f32, 2]]' to 'ListLiteral[[]]'
fn len[*types: AnyType](value: ListLiteral[types]) -> Int:
^
/.modular/Kernels/mojo/Stdlib/Len.mojo:106:1: candidate not viable: argument #0 cannot be converted from 'DynamicVector[SIMD[f32, 2]]' to 'TupleLiteral[[]]'
fn len[*types: AnyType](value: TupleLiteral[types]) -> Int:
^
/.modular/Kernels/mojo/Stdlib/Len.mojo:118:1: candidate not viable: callee expects 1 input parameter but 0 were provided
fn len[size: Int](value: StaticIntTuple[size]) -> Int:
When casting a value to a different type that alters its representation, if the original identifier is used again, the value of the new identifier will be as expected when printed alone. However, when printing it alongside another value, it will display the modified value from the initial cast.
On the playground:
from DType import DType
let a: UI64 = 999999999999
let b = a.cast[DType.ui32]()
let c = a.to_int()
print(c)
print("c:", c)
print(a, c)
999999999999
c: -727379969
999999999999 -727379969
Hash the username/email that's used on the Mojo playground notebook server URLs, instead of parsing it as it.
As of now, the Mojo playground service URLs have the username/email directly embedded in them. For eg. https://playground.modular.com/user/[email protected]/lab/tree/HelloMojo.ipynb
has the account email [email protected]
in the URL itself.
Instead, this issue proposes to use the hash of the account email/username, or perhaps even the hash of an internal account id etc. for 3 fold benefits, that I can think of now:
Add support for Apache Arrow to enable usage of all the tools that come with it, potentially via arrow2
Apache Arrow and pyarrow are big parts of the python data community, support for arrow datasets would enable interop with all the python libraries that use it.
It would also enable mojo users to use existing parquet libraries with little to no overhead costs since parquet readers will often deeserialize to the arrow format
The spec is described here: https://arrow.apache.org/docs/format/Columnar.html
A few DType methods would be nice such as:
min_number[T: DType]() -> Int
max_number[T: DType]() -> Int
bits[T: DType]() -> Int
infinity[T: DType]() -> SIMD[T, 1]
for floating point typesnan[T: DType]() -> SIMD[T, 1]
for floating point typesSince you have a repository on an open source platform (github) promoting mojo I think it's fair to answer the below questions here (instead of Discord):
Reason for asking is to prevent future lock-ins (people migrating away from python and finding themselves with a limited version or having to pay for mojo).
Many thanks,
Max
I'm curious what the Mojo team's thoughts are on enum destructuring (once enums are implemented), especially outside of match
statements.
For a language with enums (a.k.a. "variants" or "sum types"), pattern matching and destructuring is ubiquitous. The traditional way to perform a pattern match is using a match
statement. However, this is too heavy-handed for many use-cases. Accordingly, modern PLs such as Swift and Rust have lightweight syntaxes for destructuring a value within an if
or while
condition. I expect that Mojo will have a similar syntax, so I thought I'd initiate a discussion about it.
Rust has a nice approach (inspired by Swift) for testing whether a value is of a certain variant within an if
or while
condition:
if let Some(x) = iter.next() {...}
The ability to chain this destructuring is in development. It would allow statements such as:
if let Some(x) = iter.next() && let Some(y) = x.foo() {...}
This is a very useful feature, but the syntax is a bit awkward to read. Some Rust contributors proposed an alternative, easier-to-read syntax:
if iter.next() is Some(x) && x.foo() is Some(y) {...}
The challenge with the latter syntax is making it obvious that x
and y
are newly-bound variables. One solution would be to mark them as such. Using Mojo's syntax that might look like:
if iter.next() is Some(var x) and x.foo() is Some(var y) ...
As a bonus, such a syntax could allow users to specify the manner in which they want to access the value:
if iter.next() is Some(inout x) and x.foo() is Some(borrowed y) ...
(I'm not sure whether the exact keywords that I've used here make sense... I'm just trying to paint a broad picture.)
Also, having to explicitly declare each variable using var
etc. would allow existing variables to be used within a pattern, something that many languages don't support:
let x = Leaf
let tree = Node(3, Leaf, Leaf)
if tree is Node(3, x, var subtree):
...
Another interesting consideration is whether these conditions should be usable as Boolean expressions. Only expressions that don't introduce variables would be permissible:
let condition = iter.next() is Some(_)
Finally, Rust also has a nice construct known as let-else, which allows the user to effectively assert that a value matches a certain variant. If it doesn't, the current scope must be exited. In Mojo's syntax, this might look like:
iter.next() is Some(var x) else return
I haven't discussed unconditional destructuring here (e.g. destructuring a tuple), but I imagine that its design would be consistent with that of conditional destructuring.
Side note: I haven't thought much about whether destructuring operator should be is
, ==
, or a new keyword. It's possible that is
isn't the right choice here.
So, is a syntax such as this in the works? A really practical and intuitive design seems possible.
In Rust, shadowing of let bindings are allowed. I have found this feature relatively nice to have.
One example is casting data types:
fn my_fn(i: F32):
let i = I32(i)
print(i)
edit: def
-> fn
to make clear this is about stricter semantics
I see strings are defined as mutable dynamic vectors of si8 and null terminated for C compatibility. Is that the plan, or temporary? Is it planned that strings be immutable Unicode like Python 3? Thank you.
When you attempt to access a key that does not exist in a Python dict, the raised KeyError
seems to have an invalid pointer for its message (.value
property), resulting in either a missing error message or a segfault if you attempt to display it.
(This seems to only apply to KeyError
. It does not apply to the IndexError
you get if you try to access a missing index of a list; that message seems to be fine.)
from PythonInterface import Python
let python_builtins = Python.import_module("builtins")
let dict = python_builtins.dict;
var my_dict: PythonObject = dict()
try:
_ = my_dict["this key does not exist"]
except err:
print(err.value)
error: Execution was interrupted, reason: signal SIGSEGV: invalid permissions for mapped object (fault address: 0x555f68e42810).
The process has been left at the point where it was interrupted, use "thread return -x" to return to the state before expression evaluation.
Slightly modifying the code can produce different results. The following results in the error message pointer being valid instead of segfaulting, but instead of being a pointer to the error message it seems to be a pointer to the key, maybe in an argument list or something. So this seems like a memory-safety bug that could have security implications.
from PythonInterface import Python
let python_builtins = Python.import_module("builtins")
let dict = python_builtins.dict;
var my_dict: PythonObject = dict()
_ = python_builtins.print("hello, world!")
try:
_ = my_dict["this key does not exist"]
except err:
print(err.value)
hello, world!
('this key does not exist',)
11:freezer:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e535941_2d66_44e7_8688_63b8e5dff1f2.slice/cri-containerd-e4338ee9c79d388a86b863a5b887ada51bdc5678f756e5a8f204bffd3210c573.scope
10:pids:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e535941_2d66_44e7_8688_63b8e5dff1f2.slice/cri-containerd-e4338ee9c79d388a86b863a5b887ada51bdc5678f756e5a8f204bffd3210c573.scope
9:hugetlb:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e535941_2d66_44e7_8688_63b8e5dff1f2.slice/cri-containerd-e4338ee9c79d388a86b863a5b887ada51bdc5678f756e5a8f204bffd3210c573.scope
8:net_cls,net_prio:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e535941_2d66_44e7_8688_63b8e5dff1f2.slice/cri-containerd-e4338ee9c79d388a86b863a5b887ada51bdc5678f756e5a8f204bffd3210c573.scope
7:blkio:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e535941_2d66_44e7_8688_63b8e5dff1f2.slice/cri-containerd-e4338ee9c79d388a86b863a5b887ada51bdc5678f756e5a8f204bffd3210c573.scope
6:cpu,cpuacct:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e535941_2d66_44e7_8688_63b8e5dff1f2.slice/cri-containerd-e4338ee9c79d388a86b863a5b887ada51bdc5678f756e5a8f204bffd3210c573.scope
5:cpuset:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e535941_2d66_44e7_8688_63b8e5dff1f2.slice/cri-containerd-e4338ee9c79d388a86b863a5b887ada51bdc5678f756e5a8f204bffd3210c573.scope
4:devices:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e535941_2d66_44e7_8688_63b8e5dff1f2.slice/cri-containerd-e4338ee9c79d388a86b863a5b887ada51bdc5678f756e5a8f204bffd3210c573.scope
3:perf_event:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e535941_2d66_44e7_8688_63b8e5dff1f2.slice/cri-containerd-e4338ee9c79d388a86b863a5b887ada51bdc5678f756e5a8f204bffd3210c573.scope
2:memory:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e535941_2d66_44e7_8688_63b8e5dff1f2.slice/cri-containerd-e4338ee9c79d388a86b863a5b887ada51bdc5678f756e5a8f204bffd3210c573.scope
1:name=systemd:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e535941_2d66_44e7_8688_63b8e5dff1f2.slice/cri-containerd-e4338ee9c79d388a86b863a5b887ada51bdc5678f756e5a8f204bffd3210c573.scope
0::/
Here are some potential docs fixes I found. I'm not sure if they're all correct so feel free to ignore any that are unwanted.
When we started Modular, we had no intentions of building a new programming language.
"intentions" → "intention"
The journey there gives good confidence we can do this right for the Python community.
"gives good" → "gives us good"
... where incremental work put in for migration will yield incremental benefit.
"benefit" → "benefits"
For example, Mojo provides a backtick feature that allows use of any keyword as an identifier, ...
"allows use" → "allows the use"
Python is the dominant force in both the field ML and also ...
"field ML" → "ML field"
Python supports development of beautiful and expressive APIs ...
"supports development" → "supports the development"
... which led machine learning frameworks like TensorFlow and PyTorch embraced Python as ...
"embraced" → "to embrace"
More subjectively, we feel that Python is a beautiful language - designed with simple and composable abstractions, eschews needless punctuation that is redundant-in-practice with indentation, and built with powerful (dynamic) metaprogramming features ...
"and built" → "and is built"
Unfortunately, while this approach is an effective way to building high performance Python libraries, ...
"building" → "build"
The challenges with these approaches is that ...
"challenges" → "challenge"
Location: Overloaded functions and methods
When resolving a function call, Mojo tries each candidate and use the one that works (if only one works), or it picks the closest match (if it can determine a close match), or it reports that the call as ambiguous if it can’t figure out which one to pick.
"use" → "uses"
"as ambiguous" → "is ambiguous"
Location: Using parameterized types and functions
fn funWithSIMD():
# Make a vector of 4 floats.
let smallVec = SIMD[DType.f32, 4](1.0, 2.0, 3.0, 4.0)
# Make a big vector containing 1.0 in bfloat16 format.
let bigVec = SIMD[DType.bf16, 32].splat(1.0)
# Do some math and convert the elements to float32.
let biggerVec = (bigVec+bigVec).cast[DType.f32]()
# You can write types out explicitly if you want of course.
let biggerVec2 : SIMD[DType.f32, 32] = biggerVec
Maybe these variables should use snake case to align with the standard Python style (small_vec
, big_vec
, bigger_vec
, bigger_vec_2
).
Location: The __copyinit__
and __moveinit__
special methods
This is more control than languages like Swift and Rust, which require values to at least be movable.
I think "offer" should be inserted after "Rust".
Location: Parameterization: compile time meta-programming
We have currently decided to reclaim the word “parameter”, “parameter expression” to mean compile time value, ...
Insert an "and" after the comma, like this:
word “parameter”, and “parameter expression” to mean compile time value, ...
Location: Autotuning / Adaptive compilation
best_idx = f_idx
I think this is supposed to be best_idx = i
.
Location: Why argument conventions are important
e.g. “
__iadd__"
? How does”let"
work and ...
The double quotes could be fixed here, but it might look even better if they were removed altogether, like this:
e.g.
__iadd__
? How doeslet
work and ...
There are several other occurrences of double quotes around inline code elements on this page where I think the double quotes could be removed since they seem somewhat redundant, but that may just be my personal taste:
Location: fn
definitions
object
”Location: Defining parameterized types and functions
SIMD[type, size]
”Location: “Borrowed” argument convention
print_id
”const&
”const&
”use_something_big
”Int
”, “Float
”, and “SIMD
”@register_passable
”Location: “Owned” argument convention and postfix ^
operator
e.g., our
MyString
type from earlier my be defined as:
"my" → "may"
Location: @register_passable
struct decorator
it still needs to have a
__copyinit__
method to be copyable, may still have a__init__
and__del__
methods, etc.
This reads strangely because the "a" is singular but "methods" is plural. I think the "a" can be removed:
it still needs to have a
__copyinit__
method to be copyable, may still have__init__
and__del__
methods, etc.
Also in this section:
- instances of
@register_passable
types do not have predictable identity, and so the ‘self’ pointer is not stable/predictable (e.g. in hash tables).
Maybe "have predictable" should be "have a predictable".
@register_passable
arguments and result are exposed to C and C++ directly, instead of being passed by-pointer.
A space can be added between @register_passable
and "arguments". And maybe "result" should be changed to "results" or "the result" to make it read more smoothly.
- The
__init__
and__copyinit__
methods of this type are implicitly static (like new in Python) and returns its result by-value instead of takingself&
.
I think "and returns its result by-value" should be "and they return their results by-value".
Location: “Value Lifecycle”: Birth, life and death of a value
... we can look at how to put together together to model important types ...
I think "together together" is supposed to be "them together".
Location: Non-movable and non-copyable types
If we take a step up the ladder of sophistication, we’ll get to types that can be instantiated, but once they are pinned to an address in memory and cannot be implicitly moved or copied.
I think "and cannot" is supposed to be "they cannot".
Location: Unique “move-only” types
fn __moveinit__(self&, consuming existing: Self):
I think consuming existing: Self
is supposed to be owned existing: Self
. This occurs a second time on this page in the Types that support a “stealing move” section: fn __moveinit__(self&, consuming existing: Self): # as above
Also in this section, I think the code comment # This is the new.
was supposed to be something like # This is the new key capability.
(which is what the comment is in the example slightly below it.
Also in this section:
This is because instances of FileDescriptor may exist at different locations, and they can be logically moved around - stealing the body of one value and moving it another.
"it another" → "it to another"
Location: Copyable types
Each of these approaches has different tradeoffs, and Mojo takes the opinion that while we want a few common sets of collection types, that we can also support a wide range of specialized ones that focus on particular use cases.
"types, that we" → "types, we"
It implements the
__copyinit__
, which maintains the invariant that each instance of MyString owns their underlying pointer and frees it on destruction.
"owns their" → "owns its"
Mojo destroys values eagerly, which allows it to use frequently transform copy+destroy pairs into a move operation, ...
"to use frequently transform" → "to frequently transform"
Location: @value
decorator
There is no way to suppress generation of specific methods or customize generation at this time, ...
"suppress generation" → "suppress the generation"
Location: Behavior of destructors
Destroying values at the end of scope in C++ is problematic ...
Maybe "end of scope" should be "end of the scope".
Location: @parameter
decorator
A particular aspect of this feature is that it allows closures that captures runtime values to be passed as parameter values.
"captures" → "capture"
Location: Field lifetimes of owned
arguments in __del__
and __moveinit__
In the section title, a space can be added between __del__
and "and".
Also in this section, there are two places where consume(^str1)
is used, but I thought the ^
was supposed to be a postfix operator unless I'm confused and this is different than the postfix-^
"consume" operator.
Also in this section:
In this case, if “consume” implicitly refers to some value in str2 somehow,
this will ensure that str2 isn’t destroyed until the last use when it is accessed by the _ pattern.
Remove the extra line breaks between "somehow," and "this".
Location: Direct access to MLIR
Please take a look at the Low level IR in Mojo to learn how to use the
__mlir_type
,__mlir_op
,__mlir_type
constructs.
I'm not sure, but I think the ending of this sentence was supposed to be "__mlir_type
, __mlir_op
, and __mlir_attr
constructs".
All of the builtins and standard library is implemented by just calling ...
Maybe "and standard library" should be "and the standard library".
Not only is Mojo great for writing high-performance code, but it also allows us to leverage huge Python ecosystem of libraries and tools.
"huge" → "the huge"
It can also be seen as an acknowledgement of ...
"acknowledgement" → "acknowledgment"
The later approach worked well for Swift ...
"later" → "latter"
Mojo aliases can refer to parametric values but themselves cannot be parameter.
"parameter" → "a parameter"
The rememdy is to manually “rebind” the type of x, ...
"rememdy" → "remedy"
... because you pass-by-value compact struct types without indirectin via the stack.
"indirectin" → "indirection"
@register_passable
struct IntTriple:
var x: Int
var y: Int
var z: Int
fn take_int_pair(v: IntPair):
pass
let v = IntPair{x: 1, y: 2, z: 3} # no address -- it isn't on the stack!
take_int_pair(v) # copy all elements
take_int_pair
→ take_int_triple
IntPair
→ IntTriple
For instance, DynamicVector will not raise an out-of-bounds accesses (it will crash),
"accesses" → "access"
We will circle back to this when more language features and language-level optimizations are avaiable.
"avaiable" → "available"
Locaion: simd_strided_load
Perform a strided store of the SIMD vector.
"store" → "load"
The stride between stores.
"stores" → "loads"
Location: PrefetchOptions
possible locality values are:
NoLocality
,LowLocality
,MediumLocality
,HighLocality
, andVeryHighLocality
.
I'm not sure if this is anything that needs to be fixed, but it seemed odd to me so I thought I'd mention it here just in case. The docs mention the possible VeryHighLocality
value, but the highest PrefetchLocality
alias is HIGH
and the highest locality-related method is high_locality()
, neither having a "very high" variant. So I wasn't sure if this inconsistency was intentional or not.
Also, when viewing the docs on an iPad using Chrome, toggling the light/dark-theme switch resets the scroll position to the top of the page.
When asking Mojo to render a (moderately) higher resolution Mandelbrot set on the SaaS playground, the process gets killed with SIGSTOP.
xn
and yn
values toalias xn = 450*4
alias yn = 375*4
11:freezer:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1d9699f_1da3_4328_b360_63147b8b29e8.slice/cri-containerd-1a8ccbfcaad7998d860b64649a3ad7e4f6f2c676694847e674ec2ae03b618b20.scope
10:pids:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1d9699f_1da3_4328_b360_63147b8b29e8.slice/cri-containerd-1a8ccbfcaad7998d860b64649a3ad7e4f6f2c676694847e674ec2ae03b618b20.scope
9:hugetlb:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1d9699f_1da3_4328_b360_63147b8b29e8.slice/cri-containerd-1a8ccbfcaad7998d860b64649a3ad7e4f6f2c676694847e674ec2ae03b618b20.scope
8:net_cls,net_prio:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1d9699f_1da3_4328_b360_63147b8b29e8.slice/cri-containerd-1a8ccbfcaad7998d860b64649a3ad7e4f6f2c676694847e674ec2ae03b618b20.scope
7:blkio:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1d9699f_1da3_4328_b360_63147b8b29e8.slice/cri-containerd-1a8ccbfcaad7998d860b64649a3ad7e4f6f2c676694847e674ec2ae03b618b20.scope
6:cpu,cpuacct:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1d9699f_1da3_4328_b360_63147b8b29e8.slice/cri-containerd-1a8ccbfcaad7998d860b64649a3ad7e4f6f2c676694847e674ec2ae03b618b20.scope
5:cpuset:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1d9699f_1da3_4328_b360_63147b8b29e8.slice/cri-containerd-1a8ccbfcaad7998d860b64649a3ad7e4f6f2c676694847e674ec2ae03b618b20.scope
4:devices:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1d9699f_1da3_4328_b360_63147b8b29e8.slice/cri-containerd-1a8ccbfcaad7998d860b64649a3ad7e4f6f2c676694847e674ec2ae03b618b20.scope
3:perf_event:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1d9699f_1da3_4328_b360_63147b8b29e8.slice/cri-containerd-1a8ccbfcaad7998d860b64649a3ad7e4f6f2c676694847e674ec2ae03b618b20.scope
2:memory:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1d9699f_1da3_4328_b360_63147b8b29e8.slice/cri-containerd-1a8ccbfcaad7998d860b64649a3ad7e4f6f2c676694847e674ec2ae03b618b20.scope
1:name=systemd:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1d9699f_1da3_4328_b360_63147b8b29e8.slice/cri-containerd-1a8ccbfcaad7998d860b64649a3ad7e4f6f2c676694847e674ec2ae03b618b20.scope
0::/
Add wasm and wasi compile targets
Same reason why everyone is doing it. I am on mobile, I might elaborate further, if no one else has. Or you can nuke this one.
TBD
The ability to have high level layers kinda like in the keras.layers
api to kinda do something like layers.Dense(32, activation='relu')
or pytorch.nn
with torch.nn.Linear(5, 32)
this will even reduce the barrier to entry and already have optimised functions for these basic building blocks of nn's. although there will probably be arguments about this if the community should creates libraries like these
have all the building blocks of a NN in high level functions to make it easier for users to use
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.