Posted by Paris Hsu – Product Supervisor, Android Studio

At each stage of the event lifecycle, Gemini in Android Studio has grow to be your AI-powered companion, making it simpler to construct top quality apps. We’re excited to announce a big growth: Gemini in Android Studio now helps multimodal inputs, which helps you to connect photos on to your prompts! This unlocks a wealth of recent potentialities that enhance crew collaboration and UI growth workflows.

You possibly can check out this new function by downloading the newest Android Studio canary. We’ve outlined just a few use circumstances to strive, however we’d love to listen to what you suppose as we work by bringing this function into future steady releases. Test it out:

Picture attachment – a brand new dimension of interplay

We first previewed Gemini’s multimodal capabilities at Google I/O 2024. This know-how permits Gemini in Android Studio to grasp easy wireframes, and rework them into working Jetpack Compose code.

You will now discover a picture attachment icon within the Gemini chat window. Merely connect JPEG or PNG recordsdata to your prompts and watch Gemini perceive and reply to visible data. We have noticed that photos with sturdy colour contrasts yield the very best outcomes.

We encourage you to experiment with numerous prompts and pictures. Listed below are just a few compelling use circumstances to get you began:

• Fast UI prototyping and iteration: Convert a easy wireframe or high-fidelity mock of your app’s UI into working code.
• Diagram clarification and documentation: Acquire deeper insights into advanced structure or information circulation diagrams by having Gemini clarify their parts and relationships.
• UI troubleshooting: Seize screenshots of UI bugs and ask Gemini for options.

Fast UI prototyping and iteration

Gemini’s multimodal assist permits you to convert visible designs into practical UI code. Merely add your picture and use a transparent immediate. It really works whether or not you are working from your individual sketches or from a designer mockup.

Right here’s an instance immediate: “For this picture supplied, write Android Jetpack Compose code to make a display screen that is as near this picture as attainable. Be certain to incorporate imports, use Material3, and doc the code.” After which you’ll be able to append any particular or extra directions associated to the picture.

For extra advanced UIs, refine your prompts to seize particular performance. For example, when changing a calculator mockup, including “make the interactions and calculations work as you’d count on” leads to a totally practical calculator:

Be aware: this function offers an preliminary design scaffold. It’s a very good “first draft” and your edits and changes shall be wanted. Widespread refinements embody making certain right drawable imports and importing icons. Take into account the generated code a extremely environment friendly start line, accelerating your UI growth workflow.

Diagram clarification and documentation

With Gemini’s multimodal capabilities, you may also strive importing a picture of your diagram and ask for explanations or documentation.

Instance immediate: Add the Now in Android structure diagram and say “Clarify the parts and information circulation on this diagram” or “Write documentation about this diagram”.

UI troubleshooting

Leverage Gemini’s visible evaluation to establish and resolve bugs shortly. Add a screenshot of the problematic UI, and Gemini will analyze the picture and counsel potential options. You may also embody related code snippets for extra exact help.

Within the instance beneath, we used Compose UI examine and located that the button is stretched too huge in pill screens, so we took a screenshot and requested Gemini for options – it was capable of leverage the window dimension courses to offer the appropriate repair.

Obtain Android Studio immediately

Obtain the newest Android Studio canary immediately to strive the brand new multimodal options!

As all the time, Google is dedicated to the accountable use of AI. Android Studio will not ship any of your supply code to servers with out your consent. You possibly can learn extra on Gemini in Android Studio’s dedication to privateness.

We respect any suggestions on belongings you like or options you wish to see. For those who discover a bug, please report the problem and likewise try recognized points. Keep in mind to additionally comply with us on X, Medium, or YouTube for extra Android growth updates!