Adobe has been investing in generative AI tech for creative work for years, and now researchers have started using AI models to enable small, precise edits — the very opposite of an AI hallucination, if you will.
These are three of the projects that will be presented at Adobe Max during its Sneaks session on Wednesday. You can follow along with CNET’s live updates, but this is your exclusive sneak peek.
Project Deep Brush for context-aware painting
Project Deep Brush, developed by senior research scientist Iliyan Georgiev, introduces a context-aware digital brush that paints seamlessly across uneven surfaces such as walls, fabric and skin. Instead of applying text, paint or objects as flat, separate layers, the tool automatically bends and conforms them to the scene’s natural geometry and physics. Automating this process eliminates a task that would otherwise require significant time and advanced manual skill.
The tool automatically matches the lighting of an image, so the shadows make logical sense. You can have the painting curve behind existing objects, like a tattoo that disappears under a shirt collar. You can select between different modes, including flat, tilt and warp, to best align with the physical geometry of objects in a project.
Project Hands-On for wordless AI prompting
While AI enthusiasts might enjoy writing the perfect AI prompt, Project Hands-On lets creators literally get their hands on the objects in a scene with a new dynamic user interface.
Celso Gomes, principal research designer, told me that it’s a short two-step process. The program runs an image analysis on the project, identifying objects and their characteristics in the scene. The AI pulls up a short list of widgets for elements that you can adjust — an analysis of a perfume bottle would include the bottle shape, the material it’s made of, and the color and the photo style, for example. Then, using the dynamic UI placed over objects, you can resize and edit different components of each object until you get it just right.
You can apply multiple edits in one go before running the changes, a nice time saver for professionals who don’t want to wait in between each generation. Behind the program is a hodgepodge of AI models, combining UI and AI image knowledge into a new editing tool.
Project Scene Sweep for precise multishot video edits
Project Scene Sweep is one of the most impressive AI video tools I’ve encountered. With this tool, you can upload your raw film, select a character, write a prompt to adjust something (change their hairstyle or jacket, for example), and the AI will identify the character in every shot and apply the change consistently across the entire timeline. And you can make multiple edits.
Hyeonho Jeong, a research engineer presenting the project, told me the video editor isn’t meant to change the narrative of a piece. Instead, it’s meant to help editors make use of the content they have and avoid costly reshoots.
Big picture: The future of Adobe research
The throughline across all three of these projects is using AI for precise edits. Generative AI, as a whole, isn’t always the most precise tool — that’s how we end up with things like hallucinations and seven-fingered people in AI images. But new advancements in the tech over the past year, evident in these projects, have helped bring that level of control and precision to creators.
The goal is to create tech that understands the user’s intent, said Hailin Jin, senior principal scientist who oversees image and design research. “This is not just only about getting the best or most interesting adjustments to a particular image,” Jin said. “It’s about streamlining the entire process so that users don’t have to be working on repetitive things… so users can focus on the creative part.”
There’s no guarantee any of the Sneaks projects will be released to the public, though two of these, Deep Brush and Hands-On, are getting an initial public release this week on the relaunched Adobe Labs website. The Adobe Labs playground space will grant early access to select research previews going forward.
According to Gautham Mysore, senior principal scientist overseeing video and audio research, the goal is finding new ways to blend science and art.
“Building out new technical methodology, the science and the art together,” said Mysore. “That’s the path we’re gonna go.”
