New ML framework from Apple called MLX

[tomO2013]

Another interesting feature request for complex64 metal gemm...

Add Gemm for complex64 · Issue #140 · ml-explore/mlx

Not sure if this is an issue of mine, but I'm unable to use mx.matmul or use the @ operator on two mx.arrays, it throws the following error: ` libc++abi: terminating due to uncaught exception of ty...

github.com

A different MPS backend isn't a huge tradeoff (and probably preferable as it will be more stable today in general - I wouldn't really look to MLX for anything mission critical important right now - more something I'd like to tinker with). The performance differential between MLX and MPS isn't as huge on M3 owing to the new chip improvements as it would be with other M1/M2 silicon that if you're using MLS anyway, you're not really missing out.

 

[JRLMustang]

I wouldn't really look to MLX for anything mission critical important right now

Emphasis added.

I can’t imagine Apple releasing a framework like MLX for M1 and M2 improvements alone. I know it seems that way right now, but that’s where the puck has been. No, they have bigger plans. Obviously, MLX is in its infancy. We’ve yet to see the M3 Ultra and the M4 will undoubtedly debut in 2024. I expect MLX to have some major advances in the near future to compliment the Silicon releases. Remember… “Where the puck will be. Not where it’s been”. Just a hunch.

 

[Jimmyjames]

Someone has compiled a list of benchmark scores for MLX vs CPU vs MPS.
https://github.com/TristanBilot/mlx-benchmark

M1 Pro

Operation	mlx_gpu	mlx_cpu	mps	cpu	mps/mlx_gpu speedup	mlx_cpu/mlx_gpu speedup
MatMul	15.46	36.69	23.11	593.40	0.49	1.37
Softmax	3.99	42.10	6.76	32.29	0.69	9.55
Linear	13.31	35.31	32.76	98.05	1.46	1.65
Conv2d	63.72	2505.39	10.21	125.39	-0.84	38.32
BCE	4.29	28.79	8.50	8.49	0.98	5.71
Concat	6.31	90.50	6.43	43.79	0.02	13.35

Some nice improvements except from Conv2d and Concat.

 

[Jimmyjames]

MLX seems to iterate quite rapidly. Some nice GEMM improvements in the latest release.
Some benchmarks here: https://github.com/ml-explore/mlx/pull/424#issuecomment-1898815724

 

[Altaic]

Another interesting feature request for complex64 metal gemm...

Add Gemm for complex64 · Issue #140 · ml-explore/mlx

Not sure if this is an issue of mine, but I'm unable to use mx.matmul or use the @ operator on two mx.arrays, it throws the following error: ` libc++abi: terminating due to uncaught exception of ty...

github.com

A different MPS backend isn't a huge tradeoff (and probably preferable as it will be more stable today in general - I wouldn't really look to MLX for anything mission critical important right now - more something I'd like to tinker with). The performance differential between MLX and MPS isn't as huge on M3 owing to the new chip improvements as it would be with other M1/M2 silicon that if you're using MLS anyway, you're not really missing out.

It’s alpha (breaking changes will occur for the good of humanity), but entirely useable and quite performant. There are also still some performance cliffs, so I’d expect perf to improve significantly.

Also, they’re looking to hire kernel engineers

https://twitter.com/awnihannun/status/1749924118248743360?s=46&t=ZX5n4fpf03Mlx6yhaEC8ZA

(Awni is the project lead, with three other Apple engineers and quite a few active contributors; they’re one of the most pleasant group of open source developers I’ve come across)

 

[tomO2013]

It’s alpha (breaking changes will occur for the good of humanity), but entirely useable and quite performant. There are also still some performance cliffs, so I’d expect perf to improve significantly.

Also, they’re looking to hire kernel engineers

https://twitter.com/awnihannun/status/1749924118248743360?s=46&t=ZX5n4fpf03Mlx6yhaEC8ZA

(Awni is the project lead, with three other Apple engineers and quite a few active contributors; they’re one of the most pleasant group of open source developers I’ve come across)

Agreed - they are awesome. Super friendly and no attitude. That’s a huge contributing factor to making this a success.

 

[thenewperson]

MLX Swift is a thing now.