Hi! I will share my experience building a POC to generate 2D and 3D images for a friend's project.
One friend asked me to attend a meeting at his house to define the user story for a project he is planning. This was my first in-person meeting since COVID.
The general idea is that, during the registration process:
- The user answers some predefined questions and generates a 2D Dragon image with a predefined branding style.
- The image is generated with an AI image API creation (could be nano banana, GPT, or something else)
- Generate a 3D version of your dragon with Tripo AI and get the GLB file needed for Three.js
- Add some predefined animations on Threejs to the dragon.
The initial step was researching the available APIs and their pricing.
Image generation
First, I checked and tried to generate images. For images, you have different options. If you want to use them for free and compare the same prompt across different AI models, you can use this to compare them:
I also created a banner for my LinkedIn after some experiments trying to add the logo and all in place, without overlapping text or the image with the profile picture, and with the logo in the correct position.
Gemini also includes some songs and videos, here is the link from my LinkedIn
Continue my experiments with images, I explore now the music option. I requested to the previous image, change to use the Mexican soccer jersey, and requested some music related to the soccer world cup
linkedin.com
So sometimes, although you describe the A pose as returning to the T pose and it overlaps the image, some tips add context, for example:
- Add context/role: For example, you are a graphic designer with experience in designing new characters.
- Small and specific tasks: Don't try to modify an image with all instructions at the same time. For my panini, it was on steps: remove background, change jacket color, add some smile, change background to World Cup, and add panini. Also, I shared the current image of the last Mexican jersey.
- Add some quality criteria and a checklist
I got better results with GPT-Image-2
3D Images
I asked Claude to suggest some options for the 3D images. Another alternative to Tripo is Meshy.
Also, it's important to understand the key concepts to achieve better results with 3D images and animations. We're easier with a dragon than a person.
A-pose: is the standard initial posture in which most characters are modeled: standing, facing forward, with the arms extended downward forming roughly a 45° angle with the torso (hence the "A" shape). This position is not random: it makes it easier to place the internal skeleton later. It prevents the mesh from deforming oddly in delicate areas, such as the shoulders or armpits, as the character starts to move.
Mesh: is the visible surface of the model, a kind of "digital skin" made up of thousands of small polygons (usually triangles or squares) joined together by edges and vertices. The more polygons it has, the more detailed and smooth the character will look, but it will also be heavier to process.
Rigging consists of placing a virtual skeleton made of bones and joints inside the model. Although the viewer will never see it, this skeleton is what allows the character to be animated: when a bone is moved (for example, the arm bone), the mesh surrounding it deforms in response to that movement, just as human skin follows the real bone.
I tried one dragon and a girl, and the animations were bad, but as a tester, I was curious.
I tried to export and import to Mixamo, I got an error while exporting. And with the other model, my arms were wrong.
Also, sometimes it wasn't following the A pose and generated a T pose or multiple images inside the image.
Meshy
It's easier, and I think the rig option is better because you can set the parts needed for rigging. The credits are shared between the API and the web designer.
Here I tried me as some QA image
I saw the manual rig I added myself as a Lego, but I couldn't remove the knees.
The import from Meshy to Mixamo went smoothly, with no errors.
I requested support to remove the knees for lego animations and was added to the list of features if it becomes more popular, with at least more friendly and faster answers without my username or model ID and I got the answer faster.
This is one example with meshy:
The animation is better because the model has a cape and was animated correctly.
, my website to practice testing.
I got some errors with the animations maybe threejs animations are not very common. I got some errors from the API and for meshy I need to prepare the image for the rigging because if you don't limit the image you end with a image with the Rigging API max limit. That is on their API documentation with API parameter you limit that faces.
When using input_task_id, models with more than 300,000 faces are not supported for rigging. Please use the Remesh API to reduce the face count before rigging.
The solution was add should_remesh: true and target_polycount: 150000 directly on the API request
const res = await fetch(${MESHY_BASE}/openapi/v1/image-to-3d, {
method: 'POST',
headers: headers(),
body: JSON.stringify({
image_url: imageUrl,
model_type: 'standard',
should_texture: true,
should_remesh: true,
target_polycount: 150000
})
})
One thing I noticed is that most of the examples of 2D and 3D images were of white people. And some ninjas have the cap with holes. There may be some bias about that.
SOCIAL SHARE CARD GENERATOR