![\"\"](\"https:\/\/www.newsbeep.com\/us\/wp-content\/uploads\/2025\/11\/siri-gemini.jpg\")
<\/p>\n\t<\/p>\n
Earlier this week, Bloomberg reported<\/a> that Google and Apple are close to reaching a yearly $1 billion agreement for a version of the Gemini model that will power the revamped Siri next year.<\/p>\n But perhaps more interesting than the price tag is one factor that will actually affect everyone\u2019s experience: its architecture. Here\u2019s a look at how it will likely work.<\/p>\n Is 1.2 trillion parameters a lot?<\/p>\n According to Bloomberg\u2019s report<\/a>, Google will provide Apple with a 1.2 trillion parameter model, which will live on Apple\u2019s Private Cloud Compute servers, effectively preempting Google from accessing any of it. Privacy-wise, that\u2019s great.<\/p>\n Size-wise, a 1.2 trillion parameter model is nothing to sneeze at. However, a direct comparison with the latest and greatest competing models is quite challenging.<\/p>\n That\u2019s because in recent years, closed frontier AI labs like OpenAI, Anthropic, and Google have stopped disclosing the parameter counts of their latest flagship models. This has led to wildly varying speculation as to the true parameter count of offerings such as GPT-5, Gemini 2.5 Pro, and Claude Sonnet 4.5. Some put them below a trillion parameters, while others suggest they reach a few trillion. In reality, nobody really knows.<\/p>\n On the other hand, one thing most of these huge latest models have in common is an underlying architecture known as mixture of experts (MoE). In fact, Apple already employs a flavor of MoE on its current cloud-based model<\/a>, which is rumored to have 150 billion parameters.<\/p>\n Siri\u2019s Gemini-powered model will likely use a mixture of experts<\/p>\n In a nutshell, MoE is a technique that structures a model with multiple specialized sub-networks called \u2018experts.\u2019 For each input, only a few relevant experts are activated, which results in a faster and more computationally efficient model.<\/p>\n In other words, this allows MoE models to have very high parameter counts, while keeping inference costs much lower than if 100% of their parameters had to be activated for every input.<\/p>\n Here\u2019s another thing about models that take the MoE approach: they usually have a maximum number of active experts and a maximum number of active parameters for each input, resulting in something like this:<\/p>\n A model with 1.2 trillion total parameters might use 32 experts, with only 2\u20134 experts active per token. This means only around 75\u2013150B parameters are actually making calculations at any given moment, giving you the capacity of a massive model while keeping computational costs similar to running a much smaller model.<\/p>\n Here\u2019s a great video made by IBM that explains in more detail how MoE works:<\/p>\n<\/p>\n To be clear, there have been no reports regarding the architecture of the model that Google may provide Apple with, should they seal the deal on their reported partnership. But at 1.2 trillion parameters, it is very likely that it will require the MoE approach to run efficiently, given the alternatives available today.<\/p>\n Whether that size will be enough to keep the Gemini-powered Siri competitive with the models that will be available by the time it launches next year, is a different story.<\/p>\n Accessory deals on Amazon<\/p>\n \t\t\t\t FTC: We use income earning auto affiliate links. More.<\/a><\/p>\n
\n\t\t\t![\"Add](\"https:\/\/www.newsbeep.com\/us\/wp-content\/uploads\/2025\/11\/1762284008_569_google-preferred-source-badge-dark.png\"){kind=icon}
\n\t\t\t![\"Add](\"https:\/\/www.newsbeep.com\/us\/wp-content\/uploads\/2025\/10\/1760573832_833_google-preferred-source-badge-light.png\"){kind=icon}
\n\t\t<\/a><\/p>\n
![\"\"](\"https:\/\/www.newsbeep.com\/us\/wp-content\/uploads\/2025\/11\/nomad-banner2.jpg\")
<\/a>\t\t\t\t<\/p>\n","protected":false},"excerpt":{"rendered":"Earlier this week, Bloomberg reported that Google and Apple are close to reaching a yearly $1 billion agreement…\n","protected":false},"author":2,"featured_media":278069,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[45],"tags":[182,181,507,74],"class_list":{"0":"post-278068","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-artificialintelligence","11":"tag-technology"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/278068","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/comments?post=278068"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/278068\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media\/278069"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media?parent=278068"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/categories?post=278068"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/tags?post=278068"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}