Joint benchmarks on OCI H100 infrastructure showed 10x more concurrent users, 10x higher token throughput, and 7x more tokens served without adding GPUs
CAMPBELL, Calif., June 9, 2026 /PRNewswire/ -- WEKA, the AI data and memory infrastructure company, today announced production-scale benchmarks that show how organizations can improve the economics of long-context AI inference by serving more users and tokens on the same GPU footprint. The benchmarks show that WEKA's NeuralMesh™ platform with Augmented Memory Grid™ on Oracle Cloud Infrastructure (OCI) serves 10x more concurrent users, delivers 10x higher token throughput, and produces 7x more tokens per GPU than DRAM-only configurations without adding infrastructure. The results were validated on a nine-node OCI bare-metal H100 cluster with 100,000-token context windows.
k5:G 4=2DDlQ!#}0x>365565pDD6E#676C6?46Q :5lQs:GpDD6E!=246w@=56C`QmkAmk:>8 DC4lQ9EEADi^^>>I]AC?6HDH:C6]4@>^>65:2^|$`bca`ah^(tzp0G`0{@8@0?6H];A8n:5l~paeh__e_Q E:E=6lQW!#}6HD7@E@^(6<2x~XQ 2=ElQW!#}6HD7@E@^(6<2x~XQ ^mk^Amk^5:Gm
kAmQt?E6CAC:D6 px H@C<=@25D 2C6 AFD9:?8 4@?E6IE H:?5@HD 2?5 v!& FE:=:K2E:@? E@ ?6H =:>:ED[Q D2:5 !23=@ $6=6>[ D6?:@C 5:C64E@C[ D@7EH2C6 56G6=@A>6?E[ ~C24=6 r=@F5 x?7C2DECF4EFC6] Q%96D6 36?49>2C<D D9@H 9@H (tzpVD }6FC2=|6D9 A=2E7@C> H:E9 pF8>6?E65 |6>@CJ vC:5 @? ~rx 96=AD C6>@G6 >6>@CJ 3@EE=6?64<D D@ 4FDE@>6CD 42? DFAA@CE =2C86C[ >@C6 56>2?5:?8 :?76C6?46 H@C<=@25D H:E9@FE D:>A=J 255:?8 >@C6 v!&D]Qk^Am
kAmkDEC@?8m%9C66 ~FE4@>6D %92E r92?86 E96 |2E9 @? x?76C6?46k^DEC@?8mk^Am
kAm'2=:52E65 2E AC@5F4E:@? D42=6 @? 2 32C6\>6E2= w`__ 4=FDE6C W?:?6 ?@56D[ fa v!&D[ `__[___\kDA2?mE@<6?k^DA2?m 4@?E6IE H:?5@HD[ E9@FD2?5D @7 4@?4FCC6?E FD6CDX[ }6FC2=|6D9 H:E9 pF8>6?E65 |6>@CJ vC:5 @? ~rx 56=:G6C65ik^Am
kF= EJA6lQ5:D4Qmk=:mkDEC@?8m`_I >@C6 4@?4FCC6?E FD6CD D6CG65[ H:E9@FE 255:?8 :?7C2DECF4EFC6]k^DEC@?8m }6FC2=|6D9 H:E9 pF8>6?E65 |6>@CJ vC:5 D42=65 A2DE d[___ 4@?4FCC6?E FD6CD GD] 23@FE e__ 7@C s#p|\@?=J 4@?7:8FC2E:@?D] %9:D 6=:>:?2E6D E96 72:=FC6 4=:77 E92E 9:ED H96? 42496 D2EFC2E6D 3J 6IA2?5:?8 E96 24E:G6 42496 H@C<:?8 D6E 7C@> g]ec %:q @7 s#p| E@ agf %:q @7 FD23=6 }'|6] x? 255:E:@?[ >@C6 FD6CD A6C v!& >62?D E96 D2>6 :?G6DE>6?E DEC6E496D 7FCE96C]k^=:mk=:mkDEC@?8m`_I 9:896C kDA2?mE@<6?k^DA2?m E9C@F89AFE] |@C6 @FEAFE 7C@> 6G6CJ v!& :? E96 4=FDE6C]k^DEC@?8m ~? ~rx[ }6FC2=|6D9 H:E9 pF8>6?E65 |6>@CJ vC:5 C624965 2AAC@I] EH@ >:==:@? E@<6?D A6C D64@?5[ 4@>A2C65 E@ F?56C a__[___ 7@C E96 s#p|\@?=J 32D6=:?6] u@C AC@5F4E E62>D CF??:?8 C62=\E:>6 px 762EFC6D[ :?4=F5:?8 D62C49[ DF>>2C:K2E:@?[ 4@56 2DD:DE[ 2?5 >F=E:\EFC? 286?ED[ E96 E9C@F89AFE 56E6C>:?6D E96 46:=:?8 7@C 9@H >2?J FD6CD 42? 36 D6CG65[ 9@H 72DE 762EFC6D C6DA@?5[ 2?5 9@H >F49 C6G6?F6 E96 :?7C2DECF4EFC6 42? DFAA@CE]k^=:mk=:mkDEC@?8mfI >@C6 E@<6?D D6CG65] {@H6C 4@DE A6C kDA2?mE@<6?k^DA2?m 2E D42=6]k^DEC@?8m }6FC2=|6D9 H:E9 pF8>6?E65 |6>@CJ vC:5 D6CG65 7:G6 3:==:@? E@<6?D[ 4@>A2C65 E@ f__ >:==:@? 7@C E96 s#p|\@?=J 32D6=:?6[ :? 2 D:?8=6 @?6\9@FC[ a[c__\FD6C E6DE] u@C @C82?:K2E:@?D CF??:?8 286?E:4 H@C<7=@HD[ s#p| D2EFC2E:@? BF:6E=J 5C2:?D v!& 42A24:EJ E9C@F89 4@?DE2?E C64@>AFE2E:@?[ 4C62E:?8 2 5:C64E 9:E @? 4@DE A6C kDA2?mE@<6?k^DA2?m 2?5 #~x]k^=:mk^F=m
kAmQx?76C6?46 :D 3@EE=6?64<65 3J 9@H >F49 67764E:G6 >6>@CJ :D 2G2:=23=6 E@ v!&D[Q D2:5 {:C2? +G:36=[ rt~ @7 (tzp] Q%96D6 C6DF=ED AC@G6 E92E px kDA2?mE@<6?k^DA2?m 64@?@>:4D 2C6?VE D@=G65 3J 92C5H2C6 2=@?6j E96JVC6 D@=G65 3J 6=:>:?2E:?8 E96 >6>@CJ H2== E92E 92D 366? E96 C62= 46:=:?8 @? H92E 6I:DE:?8 92C5H2C6 42? 5@] }6FC2=|6D9 H:E9 pF8>6?E65 |6>@CJ vC:5 CF??:?8 @? ~rx 3C:?8D @C56CD @7 >28?:EF56 >@C6 E@<6?D E@ 4FDE@>6CD :? 2? 6IEC6>6=J 4@DE\677:4:6?E H2J]Qk^Am
kAmkDEC@?8m%C2?D7@C>:?8 px t4@?@>:4D H:E9 r@?E6IE |6>@CJ x?7C2DECF4EFC6k^DEC@?8mk^Am
kAmpD :?76C6?46 56>2?5 8C@HD[ px :?7C2DECF4EFC6 :?677:4:6?4:6D 4@>A@F?5] tG6CJ <6J\G2=F6 Wz'X 42496 6G:4E:@? :D 2 E2Ii @? v!& 4J4=6D[ =2E6?4J[ FD6C 6IA6C:6?46[ 2?5 E96 4@DE @7 6G6CJ kDA2?mE@<6?k^DA2?m D6CG65] u@C =@?8\4@?E6IE 2?5 286?E:4 H@C<=@25D[ H96C6 :?AFED C@FE:?6=J CF? E@ `__[___ E@<6?D @C >@C6[ E92E E2I :D ?@E 2 C@F?5:?8 6CC@C] xE :D 2 5:C64E 9:E @? E96 F?:E 64@?@>:4D @7 6G6CJ @C82?:K2E:@? CF??:?8 AC@5F4E:@? px]k^Am
kAmpF8>6?E65 |6>@CJ vC:5[ 2 42A23:=:EJ @7 }6FC2=|6D9[ D@=G6D E96 AC@3=6> 2E E96 2C49:E64EFC2= =6G6= 3J 564@FA=:?8 z' 42496 7C@> =@42= v!& >6>@CJ 2?5 DE@C:?8 :E :? 2 9:89\A6C7@C>2?46 kDA2?mE@<6?k^DA2?m H2C69@FD6 2446DD:3=6 24C@DD E96 4=FDE6C] p?J 9@DE 42? D6CG6 2?J D6DD:@? H:E9 42496 9:ED :?E24E[ 6=:>:?2E:?8 C:8:5 D6DD:@? DE:4<:?6DD H9:=6 56=:G6C:?8 DFA6C:@C A6C7@C>2?46 E@ s#p|[ :>AC@G:?8 =@25 32=2?4:?8[ 2?5 6?23=:?8 4=62? 9@C:K@?E2= D42=:?8 2D 4@?4FCC6?4J 8C@HD] %96 C6DF=E :D A6CD:DE6?E 4@?E6IE >6>@CJ 7@C px 286?ED 2?5 E96 4@DE =6G6C E92E >2<6D =@?8\4@?E6IE :?76C6?46 64@?@>:42= E@ CF? 2E D42=6]k^Am
More from this section
kAmkDEC@?8m!C@5F4E:@?\vC256 !C@@7k^DEC@?8mk^Am
kAm~rx AF3=:D965 E96 7F== 36?49>2C< >6E9@5@=@8J[ DJDE6> 4@?7:8FC2E:@?[ 2?5 C6DF=ED @? :ED k2 9C67lQ9EEADi^^6586]AC?6HDH:C6]4@>^4^=:?<^nEl_U2>Aj=l6?U2>Aj@lcf_dg`d\`U2>Aj9laafggbac_dU2>AjFl9EEADTbpTauTau3=@8D]@C24=6]4@>Tau2:\2?5\52E2D4:6?46TauD42=:?8\=@?8\4@?E6IE\:?76C6?46\@?\@4:\H:E9\H6<2D\2F8>6?E65\>6>@CJ\8C:5U2>Aj2lpxZTaeZs2E2Z$4:6?46Z3=@8Q E2C86ElQ03=2?<Q C6=lQ?@7@==@HQmpx U2>Aj s2E2 $4:6?46 3=@8k^2m @? |2J `b[ a_ae] %96 36?49>2C<D[ 6I64FE65 @? 2 ?:?6\?@56 ~rx 32C6\>6E2= w`__ 4=FDE6C[ >@G6 36J@?5 E96 AC:@C A92D6 @7 G2=:52E:@?[ H9:49 56>@?DEC2E65 k2 9C67lQ9EEADi^^6586]AC?6HDH:C6]4@>^4^=:?<^nEl_U2>Aj=l6?U2>Aj@lcf_dg`d\`U2>Aj9lfa_e`a`eU2>AjFl9EEADTbpTauTauHHH]H6<2]:@Tau4@>A2?JTauH6<2\?6HDC@@>TauAC6DD\C6=62D6DTauH6<2\3C62<D\E96\2:\>6>@CJ\32CC:6C\H:E9\2F8>6?E65\>6>@CJ\8C:5TauU2>Aj2l`___IZ>@C6Zz'Z42496Z42A24:EJZ2?5ZFAZE@Za_IZ72DE6CZE:>6ZE@Z7:CDEZE@<6?Z2EZ`agTar___ZE@<6?DQ E2C86ElQ03=2?<Q C6=lQ?@7@==@HQm`___I >@C6 z' 42496 42A24:EJ 2?5 FA E@ a_I 72DE6C E:>6 E@ 7:CDE kDA2?mE@<6?k^DA2?m 2E `ag[___ E@<6?Dk^2m] %9:D =2E6DE A92D6 E6DED E96 7F== 64@?@>:4D @7 :?76C6?46 :? AC@5F4E:@?i 4@?4FCC6?4J 56?D:EJ[ DFDE2:?65 E9C@F89AFE[ 42496 A6CD:DE6?46[ 2?5 D6CG:46 =6G6= @3;64E:G6 W${~X DE23:=:EJ H96? 56>2?5 DA:<6D F?56C 9:89 =@25]k^Am
kAmkDEC@?8mpG2:=23=6 @? ~C24=6 |2C<6EA=246k^DEC@?8mk^Am
kAm}6FC2=|6D9 H:E9 k2 9C67lQ9EEADi^^6586]AC?6HDH:C6]4@>^4^=:?<^nEl_U2>Aj=l6?U2>Aj@lcf_dg`d\`U2>Aj9laadheffgdcU2>AjFl9EEADTbpTauTauHHH]H6<2]:@TauAC@5F4ETau2F8>6?E65\>6>@CJ\8C:5TauU2>Aj2lpF8>6?E65Z|6>@CJZvC:5Q E2C86ElQ03=2?<Q C6=lQ?@7@==@HQmpF8>6?E65 |6>@CJ vC:5k^2m :D 86?6C2==J 2G2:=23=6 E@ (tzp 4FDE@>6CD 2?5 @? E96 k2 9C67lQ9EEADi^^6586]AC?6HDH:C6]4@>^4^=:?<^nEl_U2>Aj=l6?U2>Aj@lcf_dg`d\`U2>Aj9lbbhch`bfhbU2>AjFl9EEADTbpTauTau>2C<6EA=246]@C24=6]4@>Tau=:DE:?8DTau?6FC2=>6D9\3J\H6<2\H:E9\2F8>6?E65\>6>@CJ\8C:5\?6HTau@4:5`]><EAF3=:DE:?8]@4`]:25]2>222222F3>c5A22cDGJ<IKDDa?8GeA9?e;5AB4@:>DIAJ=Jab><D8C98F:2U2>Aj2l~C24=6Z|2C<6EA=246Q E2C86ElQ03=2?<Q C6=lQ?@7@==@HQm~C24=6 |2C<6EA=246k^2m[ H:E9 ~rx 2D (tzpVD 6I4=FD:G6 4=@F5 =2F?49 A2CE?6C] ~C82?:K2E:@?D CF??:?8 =@?8\4@?E6IE :?76C6?46 @? ~rx 42? 56A=@J 2 G2=:52E65[ AC@5F4E:@?\C625J 2C49:E64EFC6 E@52J] u@C >@C6 @? E96 ~rx 2?5 (tzp pF8>6?E65 |6>@CJ vC:5 36?49>2C<[ C625 E96 ~rx 3=@8i k2 9C67lQ9EEADi^^6586]AC?6HDH:C6]4@>^4^=:?<^nEl_U2>Aj=l6?U2>Aj@lcf_dg`d\`U2>Aj9lf`_ge_gecU2>AjFl9EEADTbpTauTau3=@8D]@C24=6]4@>Tau2:\2?5\52E2D4:6?46TauD42=:?8\=@?8\4@?E6IE\:?76C6?46\@?\@4:\H:E9\H6<2D\2F8>6?E65\>6>@CJ\8C:5U2>Aj2l9EEADTbpTauTau3=@8D]@C24=6]4@>Tau2:\2?5\52E2D4:6?46TauD42=:?8\=@?8\4@?E6IE\:?76C6?46\@?\@4:\H:E9\H6<2D\2F8>6?E65\>6>@CJ\8C:5Q E2C86ElQ03=2?<Q C6=lQ?@7@==@HQm9EEADi^^3=@8D]@C24=6]4@>^2:\2?5\52E2D4:6?46^D42=:?8\=@?8\4@?E6IE\:?76C6?46\@?\@4:\H:E9\H6<2D\2F8>6?E65\>6>@CJ\8C:5k^2m]k^Am
kAmkDEC@?8mp3@FE (tzpk^DEC@?8mk^Am
kAm(tzp :D E96 px 52E2 2?5 >6>@CJ :?7C2DECF4EFC6 4@>A2?J EC2?D7@C>:?8 E96 64@?@>:4D @7 286?E:4 px] xED }6FC2=|6D9™ A=2E7@C> F?:7:6D 9:89\A6C7@C>2?46 52E2 DE@C286 H:E9 6IE6?565 v!& >6>@CJ[ 8:G:?8 6?E6CAC:D6D[ px 4=@F5 AC@G:56CD[ 2?5 px 3F:=56CD 2 D:?8=6 7@F?52E:@? 7@C EC2:?:?8[ :?76C6?46[ 2?5 286?E:4 H@C<=@25D] (:E9 pF8>6?E65 |6>@CJ vC:5[ }6FC2=|6D9 6IE6?5D v!& >6>@CJ 42A24:EJ 3J `___I[ 2446=6C2E6D E:>6 E@ 7:CDE kDA2?mE@<6?k^DA2?m 3J FA E@ a_I[ 2?5 56=:G6CD `_I >@C6 4@?4FCC6?E FD6CD 7C@> E96 D2>6 v!& 7@@EAC:?E[ AC@G6? :? AC@5F4E:@? 36?49>2C<D] %CFDE65 3J b_T @7 E96 u@CEF?6 d_[ (tzp 6?23=6D @C82?:K2E:@?D E@ D42=6 px 72DE6C[ @AE:>:K6 v!& FE:=:K2E:@?[ 2?5 C65F46 E96 4@DE @7 6G6CJ kDA2?mE@<6?k^DA2?m D6CG65] {62C? >@C6 2E k2 9C67lQ9EEADi^^6586]AC?6HDH:C6]4@>^4^=:?<^nEl_U2>Aj=l6?U2>Aj@lcf_dg`d\`U2>Aj9lbhah__dhcU2>AjFl9EEATbpTauTauHHH]H6<2]:@TauU2>Aj2lHHH]H6<2]:@Q E2C86ElQ03=2?<Q C6=lQ?@7@==@HQmHHH]H6<2]:@k^2m @C 4@??64E H:E9 FD @? k2 9C67lQ9EEADi^^6586]AC?6HDH:C6]4@>^4^=:?<^nEl_U2>Aj=l6?U2>Aj@lcf_dg`d\`U2>Aj9l``cf_acffaU2>AjFl9EEADTbpTauTauHHH]=:?<65:?]4@>Tau4@>A2?JTauH6<2\:@TauU2>Aj2l{:?<65x?Q E2C86ElQ03=2?<Q C6=lQ?@7@==@HQm{:?<65x?k^2m 2?5 k2 9C67lQ9EEADi^^6586]AC?6HDH:C6]4@>^4^=:?<^nEl_U2>Aj=l6?U2>Aj@lcf_dg`d\`U2>Aj9lbc`_gce`dU2>AjFl9EEADTbpTauTauI]4@>TauH6<2U2>Aj2l)Q E2C86ElQ03=2?<Q C6=lQ?@7@==@HQm)k^2m]k^Am
kA 4=2DDlQAC?E24Qmk6>m(tzp 2?5 E96 ( =@8@ 2C6 C68:DE6C65 EC256>2C<D @7 (6<2x~[ x?4] ~E96C EC256 ?2>6D 96C6:? >2J 36 EC256>2C<D @7 E96:C C6DA64E:G6 @H?6CD]k^6>mk^Am
kA :5lQ!&#{Qmk:>8 E:E=6lQr:D:@?Q H:5E9lQ`aQ 96:89ElQ`aQ 2=ElQr:D:@?Q DC4lQ9EEADi^^6586]AC?6HDH:C6]4@>^4^:>8^72G:4@?]A?8nD?l$ufffgfU2>AjD5la_ae\_e\_hQ ^m ':6H @C:8:?2= 4@?E6?E E@ 5@H?=@25 >F=E:>65:2ik2 :5lQ!#}&#{Q C6=lQ?@7@==@HQ 9C67lQ9EEADi^^HHH]AC?6HDH:C6]4@>^?6HD\C6=62D6D^H6<2\2?5\@C24=6\4=@F5\:?7C2DECF4EFC6\G2=:52E6\`_I\E9C@F89AFE\82:?D\7@C\=@?8\4@?E6IE\2:\:?76C6?46\b_afhbfbb]9E>=Q E2C86ElQ03=2?<Qm9EEADi^^HHH]AC?6HDH:C6]4@>^?6HD\C6=62D6D^H6<2\2?5\@C24=6\4=@F5\:?7C2DECF4EFC6\G2=:52E6\`_I\E9C@F89AFE\82:?D\7@C\=@?8\4@?E6IE\2:\:?76C6?46\b_afhbfbb]9E>=k^2mk^Am
kAm$~&#rt (tzpk^Am
kAmk:>8 2=ElQQ DC4lQ9EEADi^^CE]AC?6HDH:C6]4@>^CE]8:7n}6HDxE6>x5l$ufffgfU2>Aj%C2?D>:DD:@?0x5la_ae_e_h_h__!#0}t($0&$!#00000$ufffgfU2>Ajs2E6x5la_ae_e_hQ DEJ=6lQ3@C56Ci_AIj H:5E9i`AIj 96:89Ei`AIjQ ^mk^Am
