In October 2024, IBM released Granite 3.0, the third generation of its Granite series of large language models (LLMs). The new model in the family brings in a number of exciting advances, such as the use of a Mixture of Experts architecture for greater computational efficiency and new LLM-based guardrail models delivering the most comprehensive risk and harm detection capabilities currently available. However, in this article, I want to focus specifically on the potential for using these powerful new open-source models for code generation tasks. I aim to highlight what sets them apart and why they're particularly well-suited for transforming the coding experience in the enterprise.

The excitement surrounding generative AI code generation is growing exponentially. Even if developers aren't yet using these tools regularly at work, many have already experimented with the likes of OpenAI's ChatGPT or Anthropic's Claude to generate code snippets or ideas to improve their code. According to IT industry analyst Gartner, 75% of enterprise sofware engineers are expected to be using AI code assistants by 2028.

What are the Granite code models?

In brief, Granite 3.0 is IBM's third generation of LLMs, released under an Apache 2.0 license. Trained on over 12 trillion tokens across 12 human languages and 116 programming languages, these models can be used for a variety of applications including tool use, RAG, summarization, entity extraction, classification as well as code generation, code translation, code bug fixing, and more. The Granite 3.0 family includes:

• Granite Dense - Powerful "workhorse" models designed for enterprise AI (Granite-3.0-8B-Instruct, Granite-3.0-8B-Base, Granite-3.0-2B-Instruct, Granite-3.0-2B-Base)
• Granite MoE - Inference-efficient Mixture of Experts (MoE) models ideal for on device applications or CPU-based deployments (Granite-3.0-3B-A800M-Instruct, Granite-3.0-3B-A800M-Base, Granite-3.0-1B-A400M-Instruct, Granite-3.0-1B-A400M-Base)
• Granite Accelerator - Speculative decoding companion model for increased inference speed and lower latency (Granite-3.0-8B-Instruct-Accelerator)

When it comes to AI-assisted coding, the Granite 3.0 models will build on the capabilities of the earlier Granite code gen family of models (launched in May 2024). Specifically, they will help IBM to deliver on its stated aim in the code generation space, which is to make coding "as easy as possible - for as many developers as possible", with a special emphasis on enterprise users.

The 116 programming languages in their training data include Python, Java and mainframe languages such as Assembler and COBOL, which means that the Granite 3.0 family is well suited to supporting code gen tasks within the enterprise space.

What can they do?

For code generation, Granite 3.0 models have the potential to offer a variety of capabilities that can help developers work faster and improve their productivity.

In particular, there are a number of areas that could be automated. They include code completion, in which the model predicts and fills in the next lines of code or code blocks as the developer is writing; refactoring, in which the model automatically improves code structure and readability without changing its functionality; explanation, in which the model provides human-readable explanations of code functionality (imagine the time you can save if instead of having to read 1000s of lines of code, you can get an instant summary that explains it); and bug fixing.

However, it's important to remember that AI code generation is in its infancy. The way we use models such as Granite will evolve to include new and unexpected practices. For example, I've been using AI code generation tools to help me quickly understand the underlying code - and it's been really helpful. I think this could catch on as an alternative to text-based code explanations.

Other coding capabilities for Granite 3.0 could include assisting in managing and updating code repositories and code translation - which is converting code from one programming language to another. There is also the potential for developers to work with the models to create new approaches to translating legacy codebases like COBOL into more modern language like Java, for example.

Why is Granite perfect for enterprise?

With enterprise development teams increasingly under pressure to get projects out the door faster, the speed and efficiency gains delivered by the Granite 3.0 models will be extremely valuable.

Importantly, in IBM's evaluation tests, the IBM Granite 3.0 models match - and, in some cases, exceed - the general performance of leading open weight LLMs across both academic and enterprise benchmarks. For example, the Granite 8.0B Instruct model rivals similarly sized models from Meta and Mistral AI.

In addition, in Granite 3.0, IBM has incorporated speculative decoding, an optimization technique for accelerating model inference speed, helping LLMs generate text (or code) faster while using the same (or less) compute resources. This allows more users to use the model simultaneously, providing a valuable benefit for enterprise development teams.

Importantly, the Granite 3.0 family has been trained to excel in key enterprise domains, such as cybersecurity. The Granite 3.0 8B Instruct, for example, excels on both IBM's proprietary cybersecurity benchmarks and prominent public security benchmarks.

Moreover, the Granite models are easily accessible to enterprise development teams via extensions to popular IDEs such as VS Code. This means developers can seamlessly integrate AI code generation functionality into their existing projects and workflows. They can avoid the friction of having to switch between windows, for example, as they would have to when using the likes of ChatGPT.

The different model sizes within the Granite 3.0 series have been made available to suit various use cases from those that are deployed in resource-constrained environments to complex enterprise applications.

The fact that the Granite models can be hosted within the corporate network or in the private cloud does away with concerns about an organisation's IP being exposed to the outside world. With cloud-based gen AI models or tools, many businesses are put off by the risk - albeit very small - that code or data entered into the system by their developers will become part of the model's training data and wind up being spewed out as output to other users.

Trust and protection

Among the most significant enterprise benefits of all the Granite models are those related to trust and protection, which is rooted in IBM's approach to model training.

Similar to the other models in the Granite series (which I've written about previously), the Granite 3.0 family are trained on permissively licensed data following IBM's AI ethics principles, making them safe for enterprise use.

The training includes a rigorous end-to-end process that filters out duplicates, copyrighted material, poor-quality data, and data with GDPR protections, meaning enterprises can be confident that IBM has done its utmost to remove bias or misuse from the models. To build even more trust, IBM indemnifies customers from potential copyright infringements or unintended outputs generated by the models.

Another priority for enterprise users is ensuring compliance with the evolving regulations surrounding AI, such as the EU AI Act. In this, the Granite models will be at an advantage since IBM is very proactive in AI governance, positioning itself ahead of other LLM providers in compliance readiness.

Why open source?

By making the Granite models available as open-source projects on popular platforms such as GitHub and Hugging Face, IBM is aiming to widen the potential user base. It is also hoping to encourage the global open-source community to contribute to their development. This can potentially speed up innovation, including the development of more fine-tuned models to suit different use cases and new ways to improve efficiency.

It's also true that enterprise users sometimes prefer using open-source technology because it prevents them from becoming dependent on specific vendors. They also appreciate that open-source deployments give them greater control to customize the models and integrate them into different environments.

IBM has a strong commitment to open source AI. That includes end-to-end transparency about model training, including releasing all the technical details behind the training and testing process. Everything is out in the open. So, customers can be reassured that they know exactly what they are getting.

Interestingly, industry commentators are speculating that one of the main reasons for other major AI vendors not releasing open source models is that their datasets are filled with copyrighted or other intellectual property-protected data. By opening up their data, they could potentially open themselves up to lawsuits.

Conclusion

Code Generation is an obvious use case for enterprises to start benefiting from the power of Generative AI. There are many positives about the Granite 3.0 models in this space. From the fact that IBM takes the utmost care in the training and filtering process to ensure customers are protected from any kind of bias, copyright infringement, or risk of misuse associated with the model output to the open-sourcing of the models, which is designed to drive greater collaboration and innovation around how they can be used.

This blog was originally published on IBM Community.