UI, 3D major updates

README.md

@@ -5,7 +5,7 @@ colorFrom: yellow
 colorTo: purple
 sdk: gradio
 python_version: 3.10.13
-sdk_version: 5.22.0
+sdk_version: 5.23.1
 app_file: app.py
 pinned: true
 short_description: Transform Your Images into Mesmerizing Hexagon Grids
@@ -36,15 +36,19 @@ Check out the configuration reference at https://huggingface.co/docs/hub/spaces-
 ## Description
 Welcome to HexaGrid Creator, the ultimate tool for transforming your images into mesmerizing hexagon grid masterpieces! Whether you're a tabletop game enthusiast, a digital artist, or just someone who loves unique patterns, HexaGrid Creator has something for you.
 
+### <span style='color: red; font-weight: bolder;'>ZeroGPU sometimes crashes or is not available.<br/Try Again in 10 seconds</span>
+
 ### What Can You Do?
-- **Generate Hexagon Grids:** Create stunning hexagon grid overlays on any image with fully customizable parameters. (also square and triangles)
-- **AI-Powered Image Generation:** Use AI to generate images based on your prompts and apply hexagon grids to them.
-- **Color Exclusion:** Pick and exclude specific colors from your hexagon grid for a cleaner and more refined look.
-- **Interactive Customization:** Adjust hexagon size, border size, rotation, background color, and more in real-time.
-- **Depth and 3D Model Generation:** Generate depth maps and 3D models from your images for enhanced visualization.
-- **Image Filter [Look-Up Table (LUT)] Application:** Apply filters (LUTs) to your images for color grading and enhancement.
-- **Pre-rendered Maps:** Access a library of pre-rendered hexagon maps for quick and easy customization.
-- **Add Margins:** Add customizable margins around your images for a polished finish.
+- **Generate Hexagon Grids:** Create stunning hexagon, square, or triangle grid overlays with fully customizable parameters.
+- **AI-Powered Image Generation:** Use advanced AI models and LoRA weights to generate images from your prompts and apply unique grid overlays.
+- **Color Exclusion:** Pick and exclude specific colors from your hexagon grid for improved clarity.
+- **Interactive Customization:** Adjust grid size, border size, rotation, background color, and more—all in real-time.
+- **Depth & 3D Model Generation:** Generate depth maps and interactive 3D models (with GLB and Gaussian extraction) for enhanced visualization.
+- **Image Filter [LUT] Application:** Apply advanced color grading filters with live previews using LUT files.
+- **Pre-rendered Maps:** Access a library of ready-to-use hexagon maps for quick customization.
+- **Add Margins:** Add customizable margins around your images for a polished print-ready look.
+- **Sketch Pad Integration:** Directly sketch on images to modify or replace them before further processing.
+
 
 ### Why You'll Love It
 - **Fun and Easy to Use:** With an intuitive interface and real-time previews, creating hexagon grids has never been this fun!
@@ -58,11 +62,12 @@ Welcome to HexaGrid Creator, the ultimate tool for transforming your images into
 3. **Download and Share:** Once you're happy with your creation, download it and share it with the world!
 
 ### Advanced Features
-- **Generative AI Integration:** Utilize models like `black-forest-labs/FLUX.1-dev` and various LoRA weights for generating unique images.
-- **Pre-rendered Maps:** Access a library of pre-rendered hexagon maps for quick and easy customization.
-- **Image Filter [Look-Up Table (LUT)] Application:** Apply filters (LUTs) to your images for color grading and enhancement.
-- **TRELLIS Depth and 3D Model Generation:** Create depth maps and 3D models from your images for enhanced visualization.
-- **Add Margins:** Customize margins around your images for a polished finish.
+- **Generative AI Integration:** Leverage models like `black-forest-labs/FLUX.1-dev` along with various LoRA weights for unique image generation.
+- **Pre-rendered Maps & Templates:** Quickly select from our curated collection of hexagon map designs.
+- **Image Filter (LUT) Application:** Fine-tune image color grading with advanced LUT support.
+- **TRELLIS Depth & 3D Model Generation:** Create detailed depth maps and 3D models, complete with GLB and Gaussian file extraction.
+- **Add Margins:** Fine-tune image margins for a professional finish.
+- **Sketch Pad Integration:** Use the built-in sketch pad to edit images on the fly before processing.
 
 Join the hive and start creating with HexaGrid Creator today!
 

app.py

@@ -1,4 +1,7 @@
+from ast import Str
+from tokenize import String
 import gradio as gr
+from numba.core.types import string
 import spaces
 import os
 import numpy as np
@@ -52,6 +55,7 @@ from utils.image_utils import (
     lerp_imagemath,
     shrink_and_paste_on_blank,
     show_lut,
+    apply_lut,
     apply_lut_to_image_path,
     multiply_and_blend_images,
     alpha_composite_with_control,
@@ -62,7 +66,8 @@ from utils.image_utils import (
     build_prerendered_images_by_quality,
     get_image_from_dict,
     calculate_optimal_fill_dimensions,
-    save_image_to_temp_png
+    save_image_to_temp_png,
+    combine_depth_map_with_Image
 )
 
 from utils.hex_grid import (
@@ -111,12 +116,8 @@ PIPELINE_CLASSES = {
     "FluxFillPipeline": FluxFillPipeline
 }
 
-from utils.version_info import (
-    versions_html,
-    #initialize_cuda,
-    #release_torch_resources,
-    #get_torch_info
-)
+import utils.version_info as version_info
+
 #from utils.depth_estimation import (get_depth_map_from_state)
 
 input_image_palette = []
@@ -125,6 +126,7 @@ current_lut_example_img = gr.State(constants.default_lut_example_img)
 user_dir = constants.TMPDIR
 lora_models = get_lora_models()
 selected_index = gr.State(value=-1)
+#html_versions = version_info.versions_html()
 
 image_processor: Optional[DPTImageProcessor] = None
 depth_model: Optional[DPTForDepthEstimation] = None
@@ -133,6 +135,7 @@ pipe: Optional[Union[FluxPipeline, FluxImg2ImgPipeline, FluxControlPipeline, Flu
 
 def start_session(req: gr.Request):
     print(f"Starting session with hash: {req.session_hash}")
+    session_hash = str(req.session_hash)
     user_dir = os.path.join(constants.TMPDIR, str(req.session_hash))
     os.makedirs(user_dir, exist_ok=True)
     
@@ -140,7 +143,8 @@ def start_session(req: gr.Request):
 def end_session(req: gr.Request):
     print(f"Ending session with hash: {req.session_hash}")
     user_dir = os.path.join(constants.TMPDIR, str(req.session_hash))
-    shutil.rmtree(user_dir)
+    if os.path.exists(user_dir):
+        shutil.rmtree(user_dir)
 
 # Register the cleanup function
 atexit.register(end_session)
@@ -777,7 +781,7 @@ def replace_with_sketch_image(sketch_image, replace_current_lut_example_img: boo
     if replace_current_lut_example_img:
         current_lut_example_img = sketch
     return sketch
-####################################### DEPTH ESTIMATION #######################################
+################################################################## DEPTH ESTIMATION ###############################################################################
 
 @spaces.GPU(progress=gr.Progress(track_tqdm=True))
 def load_trellis_model():
@@ -790,7 +794,7 @@ def load_trellis_model():
             TRELLIS_PIPELINE.cuda()
             # Preload with a dummy image to finalize initialization
             try:
-                TRELLIS_PIPELINE.preprocess_image(Image.fromarray(np.zeros((512, 512, 4), dtype=np.uint8)))    # Preload rembg
+                TRELLIS_PIPELINE.preprocess_image(Image.fromarray(np.zeros((518, 518, 4), dtype=np.uint8)))    # Preload rembg
             except:
                 pass
             print("TRELLIS_PIPELINE loaded\n")
@@ -961,7 +965,7 @@ def generate_3d_asset_part1(depth_image_source, randomize_seed, seed, input_imag
     # Determine the final seed using default MAX_SEED from constants
     final_seed = np.random.randint(0, constants.MAX_SEED) if randomize_seed else seed
     # Process the image for depth estimation
-    depth_img = depth_process_image(image_path, resized_width=1536, z_scale=336)
+    depth_img = depth_process_image(image_path, resized_width=1536, z_scale=208)
     #depth_img = resize_image_with_aspect_ratio(depth_img, 1536, 1536)
     
     user_dir = os.path.join(constants.TMPDIR, str(req.session_hash))
@@ -970,11 +974,19 @@ def generate_3d_asset_part1(depth_image_source, randomize_seed, seed, input_imag
     return depth_img, image_path, output_name, final_seed
 
 @spaces.GPU(duration=150,progress=gr.Progress(track_tqdm=True))
-def generate_3d_asset_part2(depth_img, image_path, output_name, seed, steps, model_resolution, video_resolution, req: gr.Request, progress=gr.Progress(track_tqdm=True)):
+def generate_3d_asset_part2(depth_img, image_path, output_name, seed, steps, model_resolution, video_resolution, depth_alpha: int, multi_mode: str, req: gr.Request, progress=gr.Progress(track_tqdm=True)):
     # Open image using standardized defaults
     image_raw = Image.open(image_path).convert("RGB")
     resized_image = resize_image_with_aspect_ratio(image_raw, model_resolution, model_resolution)
     depth_img = Image.open(depth_img).convert("RGBA")
+    resized_depth_image = resize_image_with_aspect_ratio(depth_img, model_resolution, model_resolution)
+    resized_depth_image_enhanced = apply_lut(resized_depth_image, "./LUT/Contrast.cube", 100)
+    combined_depth_img = combine_depth_map_with_Image(resized_image, resized_depth_image_enhanced, model_resolution, model_resolution, depth_alpha)
+
+    use_multi = (multi_mode != "singleimage")
+    if use_multi:
+        images = [resized_image, combined_depth_img, resized_depth_image]
+        processed_images = [TRELLIS_PIPELINE.preprocess_image(image,  max_resolution=model_resolution, remove_bg=False) for image in images]       
 
     if TRELLIS_PIPELINE is None:
         gr.Warning(f"Trellis Pipeline is not initialized: {TRELLIS_PIPELINE.device()}")
@@ -983,21 +995,38 @@ def generate_3d_asset_part2(depth_img, image_path, output_name, seed, steps, mod
         # Preprocess and run the Trellis pipeline with fixed sampler settings
         try:
             TRELLIS_PIPELINE.cuda()
-            processed_image = TRELLIS_PIPELINE.preprocess_image(resized_image, max_resolution=model_resolution, remove_bg = False)
-            outputs = TRELLIS_PIPELINE.run(
-                processed_image,
-                seed=seed,
-                formats=["gaussian", "mesh"],
-                preprocess_image=False,
-                sparse_structure_sampler_params={
-                    "steps": steps,
-                    "cfg_strength": 7.5,
-                },
-                slat_sampler_params={
-                    "steps": steps,
-                    "cfg_strength": 3.0,
-                },
-            )
+            if not use_multi:
+                processed_image = TRELLIS_PIPELINE.preprocess_image(combined_depth_img, max_resolution=model_resolution, remove_bg = False)
+                outputs = TRELLIS_PIPELINE.run(
+                    processed_image,
+                    seed=seed,
+                    formats=["gaussian", "mesh"],
+                    preprocess_image=False,
+                    sparse_structure_sampler_params={
+                        "steps": steps,
+                        "cfg_strength": 7.5,
+                    },
+                    slat_sampler_params={
+                        "steps": steps,
+                        "cfg_strength": 3.0,
+                    },
+                )
+            else:
+                outputs = TRELLIS_PIPELINE.run_multi_image(
+                    processed_images,
+                    seed=seed,
+                    formats=["gaussian", "mesh"],
+                    preprocess_image=False,
+                    sparse_structure_sampler_params={
+                        "steps": steps,
+                        "cfg_strength": 7.5,
+                    },
+                    slat_sampler_params={
+                        "steps": steps,
+                        "cfg_strength": 3.0,
+                    },
+                    mode=multi_mode,
+                )
 
             # Validate the mesh
             mesh = outputs['mesh'][0]
@@ -1035,6 +1064,7 @@ def generate_3d_asset_part2(depth_img, image_path, output_name, seed, steps, mod
 
         video = render_utils.render_video(outputs['gaussian'][0], resolution=video_resolution, num_frames=64, r=1, fov=45)['color']
         try:
+            #video_rf = render_utils.render_video(outputs['radiance_field'][0], resolution=video_resolution, num_frames=64, r=1, fov=45)['color']
             video_geo = render_utils.render_video(outputs['mesh'][0], resolution=video_resolution, num_frames=64, r=1, fov=45)['normal']
             video = [np.concatenate([video[i], video_geo[i]], axis=1) for i in range(len(video))]
         except Exception as e:
@@ -1043,7 +1073,7 @@ def generate_3d_asset_part2(depth_img, image_path, output_name, seed, steps, mod
         video_path = os.path.join(user_dir, f'{output_name}.mp4')
         imageio.mimsave(video_path, video, fps=8)
 
-        #snapshot_results = render_utils.render_snapshot_depth(outputs['mesh'][0], resolution=1280, r=1, fov=80)
+        #snapshot_results = render_utils.render_snapshot_depth(outputs['radiance_field'][0], resolution=1280, r=1, fov=80)
         #depth_snapshot = Image.fromarray(snapshot_results['normal'][0]).convert("L")
         depth_snapshot = depth_img
 
@@ -1106,7 +1136,8 @@ def extract_gaussian(state: dict, req: gr.Request, progress=gr.Progress(track_tq
 
 @spaces.GPU()
 def getVersions():
-    return versions_html()
+    #return html_versions
+    return version_info.versions_html()
 
 #generate_input_image_click.zerogpu = True
 #generate_depth_button_click.zerogpu = True
@@ -1122,15 +1153,16 @@ with gr.Blocks(css_paths="style_20250314.css", title=title, theme='Surn/beeuty',
     with gr.Row():
         gr.Markdown("""
         # HexaGrid Creator
-        ## Transform Your Images into Mesmerizing Hexagon Grid Masterpieces! ⬢
-        ### <span style='color: red; font-style: bolder;'>BEST VIEWED ON DESKTOP</span>""", elem_classes="intro", sanitize_html=False)
+        ## Transform Your Images into Mesmerizing Hexagon Grid Masterpieces with Advanced AI, 3D Depth, and Interactive Filters! ⬢
+        ### <span style='color: red; font-weight: bolder;'>BEST VIEWED ON DESKTOP – New Sketch Pad, Image Filters, and 3D Features Enabled</span>
+        """, elem_classes="intro", sanitize_html=False)
     with gr.Row():
         with gr.Accordion(" Welcome to HexaGrid Creator, the ultimate tool for transforming your images into stunning hexagon grid artworks. Whether you're a tabletop game enthusiast, a digital artist, or someone who loves unique patterns, HexaGrid Creator has something for you.", open=False, elem_classes="intro"):
             gr.Markdown ("""
 
             ## Drop an image into the Input Image and get started!
 
-        
+            ### <span style='color: red; font-weight: bolder;'>ZeroGPU sometimes crashes or is not available. It is not a code issue.</span>
 
             ## What is HexaGrid Creator?
             HexaGrid Creator is a web-based application that allows you to apply a hexagon grid overlay to any image. You can customize the size, color, and opacity of the hexagons, as well as the background and border colors. The result is a visually striking image that looks like it was made from hexagonal tiles!
@@ -1165,7 +1197,7 @@ with gr.Blocks(css_paths="style_20250314.css", title=title, theme='Surn/beeuty',
 
             Join the hive and start creating with HexaGrid Creator today!
         
-            """, elem_classes="intro")
+            """, elem_classes="intro", sanitize_html=False)
     with gr.Row():
         with gr.Column(scale=2):
             input_image = gr.Image(
@@ -1273,234 +1305,239 @@ with gr.Blocks(css_paths="style_20250314.css", title=title, theme='Surn/beeuty',
                 with gr.Row():
                     blur_button = gr.Button("Blur Input Image", elem_classes="solid")
                     blur_sketch_button = gr.Button("Blur Sketch", elem_classes="solid")
-    with gr.Row(elem_id="image_gen"):
-        with gr.Accordion("Generate AI Image (optional, fun)", open = False):
-            with gr.Row():
-                with gr.Column(scale=1):
-                    generate_input_image = gr.Button(
-                        "Generate from Input Image & Options ",
-                        elem_id="generate_input_image",
-                        elem_classes="solid"
-                    )
-                    # model_options = gr.Dropdown(
-                    #     label="Choose an AI Model*",
-                    #     choices=constants.MODELS + constants.LORA_WEIGHTS + ["Manual Entry"],
-                    #     value="Cossale/Frames2-Flex.1",
-                    #     elem_classes="solid"
-                    # )
-                    model_textbox = gr.Textbox(
-                        label="LORA/Model",
-                        value="Cossale/Frames2-Flex.1",
-                        elem_classes="solid",
-                        elem_id="inference_model",
-                        lines=2,
-                        visible=False
-                    )
-                    with gr.Accordion("Choose Image Style*", open=True):
-                        lora_gallery = gr.Gallery(
-                            [(open_image(image_path), title) for image_path, title in lora_models],
-                            label="Styles",
-                            allow_preview=False, preview=False ,
-                            columns=2,
-                            elem_id="lora_gallery",
-                            show_share_button=False,
-                            elem_classes="solid", type="filepath", 
-                            object_fit="contain", height="auto", format="png",
-                        )
-                    # Update map_options to a Dropdown with choices from constants.PROMPTS keys
+    with gr.Tabs(selected="hex_gen") as input_tabs:
+        with gr.Tab("HexaGrid Generation", id="hex_gen") as hexa_gen_tab:
+            with gr.Column(elem_classes="outline-important"):
+                with gr.Accordion("Advanced Hexagon Settings", open = False):
+                    with gr.Accordion("Hex Coloring and Exclusion", open = True):
+                        with gr.Row():
+                            color_picker = gr.ColorPicker(label="Pick a color to exclude",value="#505050")
+                            filter_color = gr.Checkbox(label="Filter Excluded Colors from Sampling", value=False,)                        
+                        exclude_color_button = gr.Button("Exclude Color", elem_id="exlude_color_button", elem_classes="solid")
+                        color_display = gr.DataFrame(label="List of Excluded RGBA Colors", headers=["R", "G", "B", "A"], elem_id="excluded_colors", type="array", value=build_dataframe(excluded_color_list), interactive=True, elem_classes="solid centered")
+                        selected_row = gr.Number(0, label="Selected Row", visible=False)
+                        delete_button = gr.Button("Delete Row", elem_id="delete_exclusion_button", elem_classes="solid")
                     with gr.Row():
-                        with gr.Column():
-                            map_options = gr.Dropdown(
-                            label="Map Options*",
-                            choices=list(constants.PROMPTS.keys()),
-                            value="Alien Landscape",
-                            elem_classes="solid",
-                            scale=0
-                            )
-                            # Add Dropdown for sizing of Images, height and width based on selection. Options are 16x9, 16x10, 4x5, 1x1
-                            # The values of height and width are based on common resolutions for each aspect ratio
-                            # Default to 16x9, 912x512
-                            image_size_ratio = gr.Dropdown(label="Image Aspect Ratio", choices=["16:9", "16:10", "4:5", "4:3", "2:1","3:2","1:1", "9:16", "10:16", "5:4", "3:4","1:2", "2:3"], value="16:9", elem_classes="solid", type="value", scale=0, interactive=True)
-                        with gr.Column():
-                            seed_slider = gr.Slider(
-                                label="Seed",
-                                minimum=0,
-                                maximum=constants.MAX_SEED,
-                                step=1,
-                                value=0,
-                                scale=0, randomize=True, elem_id="rnd_seed"
-                            )
-                            randomize_seed = gr.Checkbox(label="Randomize seed", value=False, scale=0, interactive=True)
-                    prompt_textbox = gr.Textbox(
-                        label="Prompt",
-                        visible=False,
-                        elem_classes="solid",
-                        value="Planetary overhead view, directly from above, centered on the planet’s surface, orthographic (rectangular tabletop_map) alien planet map, Battletech_boardgame scifi world with forests, lakes, oceans, continents and snow at the top and bottom, (middle is dark, no_reflections, no_shadows), looking straight down.",
-                        lines=4
-                    )
-                    negative_prompt_textbox = gr.Textbox(
-                        label="Negative Prompt",
-                        visible=False,
-                        elem_classes="solid",
-                        value="Earth, low quality, bad anatomy, blurry, cropped, worst quality, shadows, people, humans, reflections, shadows, realistic map of the Earth, isometric, text, camera_angle"
-                    )
-                    prompt_notes_label = gr.Label(
-                        "You should use FRM$ as trigger words. @1.5 minutes",
-                        elem_classes="solid centered small",
-                        show_label=False,
-                        visible=False
-                    )
-                    # Keep the change event to maintain functionality
-                    map_options.change(
-                        fn=update_prompt_visibility,
-                        inputs=[map_options],
-                        outputs=[prompt_textbox, negative_prompt_textbox, prompt_notes_label]
+                        start_x = gr.Number(label="Start X", value=20, minimum=-512, maximum= 512, precision=0)
+                        start_y = gr.Number(label="Start Y", value=20, minimum=-512, maximum= 512, precision=0)
+                        end_x = gr.Number(label="End X", value=-20, minimum=-512, maximum= 512, precision=0)
+                        end_y = gr.Number(label="End Y", value=-20, minimum=-512, maximum= 512, precision=0)
+                    with gr.Row():
+                        rotation = gr.Slider(-90, 180, 0.0, 0.1, label="Hexagon Rotation (degree)")
+                        sides = gr.Dropdown(label="Grid Shapes", info="The shapes that form grids",choices=["triangle", "square", "hexagon"], value="hexagon", allow_custom_value=False)
+                    with gr.Row():
+                        add_hex_text = gr.Dropdown(label="Add Text to Hexagons", choices=[None, "Column-Row Coordinates", "Column(Letter)-Row Coordinates", "Column-Row(Letter) Coordinates", "Sequential Numbers", "Playing Cards Sequential", "Playing Cards Alternate Red and Black", "Custom List"], value=None, allow_custom_value=False)
+                        x_spacing = gr.Number(label="Adjust Horizontal spacing", value=-14, minimum=-200, maximum=200, precision=1)
+                        y_spacing = gr.Number(label="Adjust Vertical spacing", value=3, minimum=-200, maximum=200, precision=1)
+                    with gr.Row():            
+                        custom_text_list = gr.TextArea(label="Custom Text List", value=constants.cards_alternating, visible=False,)
+                        custom_text_color_list = gr.TextArea(label="Custom Text Color List", value=constants.card_colors_alternating, visible=False)
+                    with gr.Row():
+                        hex_text_info = gr.Markdown("""
+                        ### Text Color uses the Border Color and Border Opacity, unless you use a custom list.
+                        ### The Custom Text List and Custom Text Color List are repeating comma separated lists.
+                        ### The custom color list is a comma separated list of hex colors.
+                        #### Example: "A,2,3,4,5,6,7,8,9,10,J,Q,K", "red,#0000FF,#00FF00,red,#FFFF00,#00FFFF,#FF8000,#FF00FF,#FF0080,#FF8000,#FF0080,lightblue"
+                        """, elem_id="hex_text_info", visible=False)
+                    add_hex_text.change(
+                    fn=lambda x: (
+                        gr.update(visible=(x == "Custom List")), 
+                        gr.update(visible=(x == "Custom List")),
+                        gr.update(visible=(x != None))
+                        ),
+                        inputs=add_hex_text,
+                        outputs=[custom_text_list, custom_text_color_list, hex_text_info]
                     )
-                with gr.Column(scale=2):
+                with gr.Row():
+                    hex_size = gr.Number(label="Hexagon Size", value=120, minimum=1, maximum=768)
+                    border_size = gr.Slider(-5,25,value=2,step=1,label="Border Size")
+                    fill_hex = gr.Checkbox(label="Fill Hex with color from Image", value=True)
+                with gr.Row():        
+                    background_color = gr.ColorPicker(label="Background Color", value="#000000", interactive=True)
+                    background_opacity = gr.Slider(0,100,0,1,label="Background Opacity %")
+                    border_color = gr.ColorPicker(label="Border Color", value="#7b7b7b", interactive=True)
+                    border_opacity = gr.Slider(0,100,50,1,label="Border Opacity %")
+                with gr.Row():
+                    hex_button = gr.Button("Generate Hex Grid!", elem_classes="solid", elem_id="btn-generate")
+                with gr.Row():
+                    output_image = gr.Image(label="Hexagon Grid Image", image_mode = "RGBA", elem_classes="centered solid imgcontainer", format="PNG", type="filepath", key="ImgOutput",interactive=True)
+                    overlay_image = gr.Image(label="Hexagon Overlay Image", image_mode = "RGBA", elem_classes="centered solid imgcontainer", format="PNG", type="filepath", key="ImgOverlay",interactive=True)        
+                with gr.Accordion("Grid adjustments", open=True):
                     with gr.Row():
-                        with gr.Column():
-                            generate_input_image_from_gallery = gr.Button(
-                                "Generate AI Image from Template Options",
-                                elem_id="generate_input_image_from_gallery",
+                        with gr.Column(scale=1):
+                            output_grid_tilt = gr.Slider(minimum=-90, maximum=90, value=0, step=0.05, label="Tilt Angle (degrees)")
+                            output_grid_rotation = gr.Slider(minimum=-180, maximum=180, value=0, step=0.05, label="Rotation Angle (degrees)")
+                        with gr.Column(scale=1):
+                            output_alpha_composite = gr.Slider(0,100,50,0.5, label="Alpha Composite Intensity*")
+                            output_blend_multiply_composite = gr.Slider(0,100,50,0.5, label="Multiply Intensity")
+                            output_overlay_composite = gr.Slider(0,100,50,0.5, label="Interpolate Intensity")
+
+        with gr.Tab("Image Generation (AI)", id="image_gen") as image_gen_tab:
+            with gr.Row(elem_id="image_gen"):
+                with gr.Accordion("Generate AI Image (optional, fun)", open = False):
+                    with gr.Row():
+                        with gr.Column(scale=1):
+                            generate_input_image = gr.Button(
+                                "Generate from Input Image & Options ",
+                                elem_id="generate_input_image",
                                 elem_classes="solid"
                             )
-                        with gr.Column():
-                            replace_input_image_button = gr.Button(
-                                "Replace Input Image with Template",
-                                elem_id="prerendered_replace_input_image_button",
-                                elem_classes="solid"
-                            )                       
-                    with gr.Row():
-                        with gr.Accordion("Template Images", open = False):
+                            # model_options = gr.Dropdown(
+                            #     label="Choose an AI Model*",
+                            #     choices=constants.MODELS + constants.LORA_WEIGHTS + ["Manual Entry"],
+                            #     value="Cossale/Frames2-Flex.1",
+                            #     elem_classes="solid", allow_custom_value=False
+                            # )
+                            model_textbox = gr.Textbox(
+                                label="LORA/Model",
+                                value="Cossale/Frames2-Flex.1",
+                                elem_classes="solid",
+                                elem_id="inference_model",
+                                lines=2,
+                                visible=False
+                            )
+                            with gr.Accordion("Choose Image Style*", open=True):
+                                lora_gallery = gr.Gallery(
+                                    [(open_image(image_path), title) for image_path, title in lora_models],
+                                    label="Styles",
+                                    allow_preview=False, preview=False ,
+                                    columns=2,
+                                    elem_id="lora_gallery",
+                                    show_share_button=False,
+                                    elem_classes="solid", type="filepath", 
+                                    object_fit="contain", height="auto", format="png",
+                                )
+                            # Update map_options to a Dropdown with choices from constants.PROMPTS keys
                             with gr.Row():
-                                with gr.Column(scale=2):
-                                    # Gallery from PRE_RENDERED_IMAGES GOES HERE
-                                    prerendered_image_gallery = gr.Gallery(label="Image Gallery", show_label=True, value=build_prerendered_images_by_quality(3,'thumbnail'), elem_id="gallery", 
-                                        elem_classes="solid", type="filepath", columns=[3], rows=[3], preview=False ,object_fit="contain", height="auto", format="png",allow_preview=False)
-                    with gr.Row():    
-                        image_guidance_stength = gr.Slider(label="Image Guidance Strength (prompt percentage)", info="applies to Input, Sketch and Template Image",minimum=0, maximum=1.0, value=0.85, step=0.01, interactive=True)
-    with gr.Column(elem_classes="outline-important"):
-        with gr.Accordion("Advanced Hexagon Settings", open = False):
-            with gr.Accordion("Hex Coloring and Exclusion", open = True):
-                with gr.Row():
-                    with gr.Column():
-                        color_picker = gr.ColorPicker(label="Pick a color to exclude",value="#505050")
-                    with gr.Column():
-                        filter_color = gr.Checkbox(label="Filter Excluded Colors from Sampling", value=False,)
-                        fill_hex = gr.Checkbox(label="Fill Hex with color from Image", value=True)
-                exclude_color_button = gr.Button("Exclude Color", elem_id="exlude_color_button", elem_classes="solid")
-                color_display = gr.DataFrame(label="List of Excluded RGBA Colors", headers=["R", "G", "B", "A"], elem_id="excluded_colors", type="array", value=build_dataframe(excluded_color_list), interactive=True, elem_classes="solid centered")
-                selected_row = gr.Number(0, label="Selected Row", visible=False)
-                delete_button = gr.Button("Delete Row", elem_id="delete_exclusion_button", elem_classes="solid")
-            with gr.Accordion("Hex Grid Location on Image", open = False):
-                with gr.Row():
-                    start_x = gr.Number(label="Start X", value=20, minimum=-512, maximum= 512, precision=0)
-                    start_y = gr.Number(label="Start Y", value=20, minimum=-512, maximum= 512, precision=0)
-                    end_x = gr.Number(label="End X", value=-20, minimum=-512, maximum= 512, precision=0)
-                    end_y = gr.Number(label="End Y", value=-20, minimum=-512, maximum= 512, precision=0)
-            with gr.Row():
-                rotation = gr.Slider(-90, 180, 0.0, 0.1, label="Hexagon Rotation (degree)")
-                sides = gr.Dropdown(label="Grid Shapes", info="The shapes that form grids",choices=["triangle", "square", "hexagon"], value="hexagon")            
-            with gr.Row():
-                add_hex_text = gr.Dropdown(label="Add Text to Hexagons", choices=[None, "Column-Row Coordinates", "Column(Letter)-Row Coordinates", "Column-Row(Letter) Coordinates", "Sequential Numbers", "Playing Cards Sequential", "Playing Cards Alternate Red and Black", "Custom List"], value=None)
-                x_spacing = gr.Number(label="Adjust Horizontal spacing", value=-14, minimum=-200, maximum=200, precision=1)
-                y_spacing = gr.Number(label="Adjust Vertical spacing", value=3, minimum=-200, maximum=200, precision=1)
-            with gr.Row():            
-                custom_text_list = gr.TextArea(label="Custom Text List", value=constants.cards_alternating, visible=False,)
-                custom_text_color_list = gr.TextArea(label="Custom Text Color List", value=constants.card_colors_alternating, visible=False)
-            with gr.Row():
-                hex_text_info = gr.Markdown("""
-                ### Text Color uses the Border Color and Border Opacity, unless you use a custom list.
-                ### The Custom Text List and Custom Text Color List are repeating comma separated lists.
-                ### The custom color list is a comma separated list of hex colors.
-                #### Example: "A,2,3,4,5,6,7,8,9,10,J,Q,K", "red,#0000FF,#00FF00,red,#FFFF00,#00FFFF,#FF8000,#FF00FF,#FF0080,#FF8000,#FF0080,lightblue"
-                """, elem_id="hex_text_info", visible=False)
-            add_hex_text.change(
-            fn=lambda x: (
-                gr.update(visible=(x == "Custom List")), 
-                gr.update(visible=(x == "Custom List")),
-                gr.update(visible=(x != None))
-                ),
-                inputs=add_hex_text,
-                outputs=[custom_text_list, custom_text_color_list, hex_text_info]
-            )
-        with gr.Row():
-            hex_size = gr.Number(label="Hexagon Size", value=120, minimum=1, maximum=768)
-            border_size = gr.Slider(-5,25,value=2,step=1,label="Border Size")
-        with gr.Row():        
-            background_color = gr.ColorPicker(label="Background Color", value="#000000", interactive=True)
-            background_opacity = gr.Slider(0,100,0,1,label="Background Opacity %")
-            border_color = gr.ColorPicker(label="Border Color", value="#7b7b7b", interactive=True)
-            border_opacity = gr.Slider(0,100,50,1,label="Border Opacity %")
-        with gr.Row():
-            hex_button = gr.Button("Generate Hex Grid!", elem_classes="solid", elem_id="btn-generate")
-        with gr.Row():
-            output_image = gr.Image(label="Hexagon Grid Image", image_mode = "RGBA", elem_classes="centered solid imgcontainer", format="PNG", type="filepath", key="ImgOutput",interactive=True)
-            overlay_image = gr.Image(label="Hexagon Overlay Image", image_mode = "RGBA", elem_classes="centered solid imgcontainer", format="PNG", type="filepath", key="ImgOverlay",interactive=True)        
-        with gr.Accordion("Grid adjustments", open=True):
+                                with gr.Column():
+                                    map_options = gr.Dropdown(
+                                    label="Map Options*",
+                                    choices=list(constants.PROMPTS.keys()),
+                                    value="Alien Landscape",
+                                    elem_classes="solid",
+                                    scale=0, allow_custom_value=False
+                                    )
+                                    # Add Dropdown for sizing of Images, height and width based on selection. Options are 16x9, 16x10, 4x5, 1x1
+                                    # The values of height and width are based on common resolutions for each aspect ratio
+                                    # Default to 16x9, 912x512
+                                    image_size_ratio = gr.Dropdown(label="Image Aspect Ratio", choices=["16:9", "16:10", "4:5", "4:3", "2:1","3:2","1:1", "9:16", "10:16", "5:4", "3:4","1:2", "2:3"], value="16:9", elem_classes="solid", type="value", scale=0, interactive=True, allow_custom_value=False)
+                                with gr.Column():
+                                    seed_slider = gr.Slider(
+                                        label="Seed",
+                                        minimum=0,
+                                        maximum=constants.MAX_SEED,
+                                        step=1,
+                                        value=0,
+                                        scale=0, randomize=True, elem_id="rnd_seed"
+                                    )
+                                    randomize_seed = gr.Checkbox(label="Randomize seed", value=False, scale=0, interactive=True)
+                            prompt_textbox = gr.Textbox(
+                                label="Prompt",
+                                visible=False,
+                                elem_classes="solid",
+                                value="Planetary overhead view, directly from above, centered on the planet’s surface, orthographic (rectangular tabletop_map) alien planet map, Battletech_boardgame scifi world with forests, lakes, oceans, continents and snow at the top and bottom, (middle is dark, no_reflections, no_shadows), looking straight down.",
+                                lines=4
+                            )
+                            negative_prompt_textbox = gr.Textbox(
+                                label="Negative Prompt",
+                                visible=False,
+                                elem_classes="solid",
+                                value="Earth, low quality, bad anatomy, blurry, cropped, worst quality, shadows, people, humans, reflections, shadows, realistic map of the Earth, isometric, text, camera_angle"
+                            )
+                            prompt_notes_label = gr.Label(
+                                "You should use FRM$ as trigger words. @1.5 minutes",
+                                elem_classes="solid centered small",
+                                show_label=False,
+                                visible=False
+                            )
+                            # Keep the change event to maintain functionality
+                            map_options.change(
+                                fn=update_prompt_visibility,
+                                inputs=[map_options],
+                                outputs=[prompt_textbox, negative_prompt_textbox, prompt_notes_label]
+                            )
+                        with gr.Column(scale=2):
+                            with gr.Row():
+                                with gr.Column():
+                                    generate_input_image_from_gallery = gr.Button(
+                                        "Generate AI Image from Template Options",
+                                        elem_id="generate_input_image_from_gallery",
+                                        elem_classes="solid"
+                                    )
+                                with gr.Column():
+                                    replace_input_image_button = gr.Button(
+                                        "Replace Input Image with Template",
+                                        elem_id="prerendered_replace_input_image_button",
+                                        elem_classes="solid"
+                                    )                       
+                            with gr.Row():
+                                with gr.Accordion("Template Images", open = False):
+                                    with gr.Row():
+                                        with gr.Column(scale=2):
+                                            # Gallery from PRE_RENDERED_IMAGES GOES HERE
+                                            prerendered_image_gallery = gr.Gallery(label="Image Gallery", show_label=True, value=build_prerendered_images_by_quality(3,'thumbnail'), elem_id="gallery", 
+                                                elem_classes="solid", type="filepath", columns=[3], rows=[3], preview=False ,object_fit="contain", height="auto", format="png",allow_preview=False)
+                            with gr.Row():    
+                                image_guidance_stength = gr.Slider(label="Image Guidance Strength (prompt percentage)", info="applies to Input, Sketch and Template Image",minimum=0, maximum=1.0, value=0.85, step=0.01, interactive=True)
+
+        with gr.Tab("Add Margins", id="margins") as margins_tab:
             with gr.Row():
-                with gr.Column(scale=1):
-                    output_grid_tilt = gr.Slider(minimum=-90, maximum=90, value=0, step=0.05, label="Tilt Angle (degrees)")
-                    output_grid_rotation = gr.Slider(minimum=-180, maximum=180, value=0, step=0.05, label="Rotation Angle (degrees)")
-                with gr.Column(scale=1):
-                    output_alpha_composite = gr.Slider(0,100,50,0.5, label="Alpha Composite Intensity*")
-                    output_blend_multiply_composite = gr.Slider(0,100,50,0.5, label="Multiply Intensity")
-                    output_overlay_composite = gr.Slider(0,100,50,0.5, label="Interpolate Intensity")            
-    with gr.Accordion("Add Margins (for printing)", open=False):
-        with gr.Row():
-            border_image_source = gr.Radio(label="Add Margins around which Image", choices=["Input Image", "Overlay Image"], value="Overlay Image")
-        with gr.Row():
-            mask_width = gr.Number(label="Margins Width", value=10, minimum=0, maximum=100, precision=0)
-            mask_height = gr.Number(label="Margins Height", value=10, minimum=0, maximum=100, precision=0)
-        with gr.Row():
-                margin_color = gr.ColorPicker(label="Margin Color", value="#333333FF", interactive=True)
-                margin_opacity = gr.Slider(0,100,95,0.5,label="Margin Opacity %")
-        with gr.Row():
-            add_border_button = gr.Button("Add Margins", elem_classes="solid", variant="secondary")
-        with gr.Row():
-            bordered_image_output = gr.Image(label="Image with Margins", image_mode="RGBA",  elem_classes="centered solid imgcontainer", format="PNG", type="filepath", key="ImgBordered",interactive=False, show_download_button=True, show_fullscreen_button=True, show_share_button=True)
-    accordian_3d = gr.Accordion("Height Maps and 3D (optional, fun)", open=False, elem_id="accordian_3d")
-    with accordian_3d:
-        with gr.Row():
-            depth_image_source = gr.Radio(
-                label="Depth Image Source",
-                choices=["Input Image", "Hexagon Grid Image", "Overlay Image", "Image with Margins"],
-                value="Input Image"
-            )
-        with gr.Accordion("Advanced 3D Generation Settings", open=False):
+                border_image_source = gr.Radio(label="Add Margins around which Image", choices=["Input Image", "Overlay Image"], value="Overlay Image")
             with gr.Row():
-                with gr.Column():
-                    # Use standard seed settings only
-                    seed_3d = gr.Slider(0, constants.MAX_SEED, label="Seed (3D Generation)", value=0, step=1, randomize=True)
-                    randomize_seed_3d = gr.Checkbox(label="Randomize Seed (3D Generation)", value=True)
-                with gr.Column():
-                    steps = gr.Slider(6, 36, value=12, step=1, label="Image Sampling Steps", interactive=True)
-                    video_resolution = gr.Slider(384, 768, value=480, step=32, label="Video Resolution (*danger*)", interactive=True)
-                    model_resolution = gr.Slider(512, 2304, value=1024, step=64, label="3D Model Resolution", interactive=True)
-        with gr.Row():
-            generate_3d_asset_button = gr.Button("Generate 3D Asset", elem_classes="solid", variant="secondary", interactive=False)
-        with gr.Row():
-            depth_output = gr.Image(label="Depth Map", image_mode="L", elem_classes="centered solid imgcontainer", format="PNG", type="filepath", key="DepthOutput",interactive=False, show_download_button=True, show_fullscreen_button=True, show_share_button=True, height=400)
-        with gr.Row():
-            # For display: video output and 3D model preview (GLTF)
-            video_output = gr.Video(label="3D Asset Video", autoplay=True, loop=True, height=400)
-        with gr.Accordion("GLB Extraction Settings", open=False):
+                mask_width = gr.Number(label="Margins Width", value=10, minimum=0, maximum=100, precision=0)
+                mask_height = gr.Number(label="Margins Height", value=10, minimum=0, maximum=100, precision=0)
             with gr.Row():
-                mesh_simplify = gr.Slider(0.9, 0.98, label="Simplify", value=0.95, step=0.01)
-                texture_size = gr.Slider(512, 2048, label="Texture Size", value=1024, step=512)
+                    margin_color = gr.ColorPicker(label="Margin Color", value="#333333FF", interactive=True)
+                    margin_opacity = gr.Slider(0,100,95,0.5,label="Margin Opacity %")
             with gr.Row():
-                extract_glb_btn = gr.Button("Extract GLB", interactive=False)
-                extract_gaussian_btn = gr.Button("Extract Gaussian", interactive=False)
+                add_border_button = gr.Button("Add Margins", elem_classes="solid", variant="secondary")
             with gr.Row():
-                with gr.Column(scale=2):
-                    model_output = gr.Model3D(label="Extracted 3D Model", clear_color=[1.0, 1.0, 1.0, 1.0], 
-                                            elem_classes="centered solid imgcontainer", interactive=True)
-                with gr.Column(scale=1):
-                    glb_file = gr.File(label="3D GLTF", elem_classes="solid small centered", height=250)
-                    gaussian_file = gr.File(label="Gaussian", elem_classes="solid small centered", height=250)
-                    gr.Markdown("""
-                    ### Files over 10 MB may not display in the 3D model viewer
-                    """, elem_id="file_size_info", elem_classes="intro" )
+                bordered_image_output = gr.Image(label="Image with Margins", image_mode="RGBA",  elem_classes="centered solid imgcontainer", format="PNG", type="filepath", key="ImgBordered",interactive=False, show_download_button=True, show_fullscreen_button=True, show_share_button=True)
+        with gr.Tab("3D and Depth (fun)", id="3D") as depth_tab:
+            accordian_3d = gr.Accordion("Clicking here to toggle between Image Generation and 3D models", open=False, elem_id="accordian_3d")
+            with accordian_3d:
+                with gr.Row():
+                    depth_image_source = gr.Radio(
+                        label="Depth Image Source",
+                        choices=["Input Image", "Hexagon Grid Image", "Overlay Image", "Image with Margins"],
+                        value="Input Image"
+                    )
+                with gr.Accordion("Advanced 3D Generation Settings", open=False):
+                    with gr.Row():
+                        with gr.Column():
+                            # Use standard seed settings only
+                            seed_3d = gr.Slider(0, constants.MAX_SEED, label="Seed (3D Generation)", value=0, step=1, randomize=True)
+                            randomize_seed_3d = gr.Checkbox(label="Randomize Seed (3D Generation)", value=True)
+                            depth_alpha = gr.Slider(-200,200,0, step=5, label="Amount of Depth Image to apply to main Image", interactive=True)
+                            multiimage_algo = gr.Radio(["singleimage","stochastic", "multidiffusion"], label="Multi-image Algorithm", value="singleimage")
+                        with gr.Column():
+                            steps = gr.Slider(6, 100, value=25, step=1, label="Image Sampling Steps", interactive=True)
+                            video_resolution = gr.Slider(384, 768, value=480, step=32, label="Video Resolution (*danger*)", interactive=True)
+                            model_resolution = gr.Slider(518, 2520, value=1540, step=28, label="3D Model Resolution", interactive=True)
+                with gr.Row():
+                    generate_3d_asset_button = gr.Button("Generate 3D Asset", elem_classes="solid", variant="secondary", interactive=False)
+                with gr.Row():
+                    depth_output = gr.Image(label="Depth Map", image_mode="L", elem_classes="centered solid imgcontainer", format="PNG", type="filepath", key="DepthOutput",interactive=False, show_download_button=True, show_fullscreen_button=True, show_share_button=True, height=400)
+                with gr.Row():
+                    # For display: video output and 3D model preview (GLTF)
+                    video_output = gr.Video(label="3D Asset Video", autoplay=True, loop=True, height=400)
+                with gr.Accordion("GLB Extraction Settings", open=False):
+                    with gr.Row():
+                        mesh_simplify = gr.Slider(0.9, 0.98, label="Simplify", value=0.95, step=0.01)
+                        texture_size = gr.Slider(512, 2048, label="Texture Size", value=1024, step=512)
+                    with gr.Row():
+                        extract_glb_btn = gr.Button("Extract GLB", interactive=False)
+                        extract_gaussian_btn = gr.Button("Extract Gaussian", interactive=False)
+                    with gr.Row():
+                        with gr.Column(scale=2):
+                            model_output = gr.Model3D(label="Extracted 3D Model", clear_color=[1.0, 1.0, 1.0, 1.0], 
+                                                    elem_classes="centered solid imgcontainer", interactive=True)
+                        with gr.Column(scale=1):
+                            glb_file = gr.File(label="3D GLTF", elem_classes="solid small centered", height=250)
+                            gaussian_file = gr.File(label="Gaussian", elem_classes="solid small centered", height=250)
+                            gr.Markdown("""
+                            ### Files over 10 MB may not display in the 3D model viewer
+                            """, elem_id="file_size_info", elem_classes="intro" )
 
     is_multiimage = gr.State(False)
     output_buf = gr.State()
@@ -1700,7 +1737,7 @@ with gr.Blocks(css_paths="style_20250314.css", title=title, theme='Surn/beeuty',
         scroll_to_output=True
     ).then(
         fn=generate_3d_asset_part2,
-        inputs=[depth_output, ddd_image_path, ddd_file_name, seed_3d, steps, model_resolution, video_resolution ],
+        inputs=[depth_output, ddd_image_path, ddd_file_name, seed_3d, steps, model_resolution, video_resolution,  depth_alpha,  multiimage_algo],
         outputs=[output_buf, video_output, depth_output],
         scroll_to_output=True
     ).then(
@@ -1762,9 +1799,8 @@ if __name__ == "__main__":
         TRELLIS_PIPELINE = TrellisImageTo3DPipeline.from_pretrained("JeffreyXiang/TRELLIS-image-large")
         TRELLIS_PIPELINE.to(device)
         try:
-           TRELLIS_PIPELINE.preprocess_image(Image.fromarray(np.zeros((512, 512, 3), dtype=np.uint8)), 512)    # Preload rembg
+           TRELLIS_PIPELINE.preprocess_image(Image.fromarray(np.zeros((512, 512, 3), dtype=np.uint8)), 512, True)    # Preload rembg
         except:
             pass
     hexaGrid.queue(default_concurrency_limit=1,max_size=12,api_open=False)
-    hexaGrid.launch(allowed_paths=["assets","/","./assets","images","./images", "./images/prerendered", 'e:/TMP'], favicon_path="./assets/favicon.ico", max_file_size="10mb")
-
+    hexaGrid.launch(allowed_paths=["assets","/","./assets","images","./images", "./images/prerendered", 'e:/TMP'], favicon_path="./assets/favicon.ico", max_file_size="10mb")

style_20250314.css

@@ -21,7 +21,7 @@
     background-color: rgba(242, 218, 163, 0.62);
 }
 
-.dark .gradio-container.gradio-container-5-22-0 .contain .intro .prose {
+.dark .gradio-container.gradio-container-5-23-1 .contain .intro .prose {
     background-color: rgba(41, 18, 5, 0.38) !important;
 }
 .toast-body.info {
@@ -165,3 +165,23 @@ a {
     padding: 2px;
     border-radius: 6px;
 }
+.selected.svelte-1tcem6n.svelte-1tcem6n {
+    font-size: large;
+    font-weight: bold;
+    color: var(--body-text-color);
+}
+.tab-wrapper.svelte-1tcem6n.svelte-1tcem6n {
+    height: var(--size-12);
+    padding-bottom: var(--size-1);
+    text-align: center;
+    background-blend-mode: multiply;
+    border-radius: var(--block-radius);
+    background-color: var(--block-background-fill);
+
+    outline-color: var(--accordion-text-color);
+    outline-style: solid;
+    outline-width: 2px;
+    outline-offset: 2px;
+    padding: 2px;
+    border-radius: 6px;
+}

trellis/pipelines/trellis_image_to_3d.py

@@ -87,6 +87,7 @@ class TrellisImageTo3DPipeline(Pipeline):
         Preprocess the input image.
         """
         # if has alpha channel, use it directly; otherwise, remove background
+        aspect_ratio = int(input.width / input.height)
         has_alpha = False
         if input.mode == 'RGBA':
             alpha = np.array(input)[:, :, 3]
@@ -114,8 +115,9 @@ class TrellisImageTo3DPipeline(Pipeline):
         size = max(bbox[2] - bbox[0], bbox[3] - bbox[1])
         size = int(size * 1.2)
         bbox = center[0] - size // 2, center[1] - size // 2, center[0] + size // 2, center[1] + size // 2
-        output = output.crop(bbox)  # type: ignore
-        output = output.resize((518, 518), Image.Resampling.LANCZOS)
+        output = output.crop(bbox)  # type: ignore        
+        new_width = round((588 * aspect_ratio) / 14) * 14
+        output = output.resize((new_width, 588), Image.Resampling.LANCZOS)
         output = np.array(output).astype(np.float32) / 255
         output = output[:, :, :3] * output[:, :, 3:4]
         output = Image.fromarray((output * 255).astype(np.uint8))
@@ -136,13 +138,15 @@ class TrellisImageTo3DPipeline(Pipeline):
             assert image.ndim == 4, "Image tensor should be batched (B, C, H, W)"
         elif isinstance(image, list):
             assert all(isinstance(i, Image.Image) for i in image), "Image list should be list of PIL images"
-            image = [i.resize((518, 518), Image.LANCZOS) for i in image]
+            aspect_ratio = int(image[0].width / image[0].height)
+            new_width = round((588 * aspect_ratio) / 14) * 14
+            image = [i.resize((new_width, 588), Image.LANCZOS) for i in image]
             image = [np.array(i.convert('RGB')).astype(np.float32) / 255 for i in image]
             image = [torch.from_numpy(i).permute(2, 0, 1).float() for i in image]
             image = torch.stack(image).to(self.device)
         else:
             raise ValueError(f"Unsupported type of image: {type(image)}")
-        
+    
         image = self.image_cond_model_transform(image).to(self.device)
         features = self.models['image_cond_model'](image, is_training=True)['x_prenorm']
         patchtokens = F.layer_norm(features, features.shape[-1:])
@@ -267,6 +271,7 @@ class TrellisImageTo3DPipeline(Pipeline):
         slat_sampler_params: dict = {},
         formats: List[str] = ['mesh', 'gaussian', 'radiance_field'],
         preprocess_image: bool = True,
+        max_resolution: int =1024,
         remove_bg: bool = True,
     ) -> dict:
         """
@@ -280,7 +285,7 @@ class TrellisImageTo3DPipeline(Pipeline):
             preprocess_image (bool): Whether to preprocess the image.
         """
         if preprocess_image:
-            image = self.preprocess_image(image, remove_bg=remove_bg)
+            image = self.preprocess_image(image, max_resolution, remove_bg=remove_bg)
         cond = self.get_cond([image])
         torch.manual_seed(seed)
         coords = self.sample_sparse_structure(cond, num_samples, sparse_structure_sampler_params)
@@ -355,6 +360,7 @@ class TrellisImageTo3DPipeline(Pipeline):
         formats: List[str] = ['mesh', 'gaussian', 'radiance_field'],
         preprocess_image: bool = True,
         mode: Literal['stochastic', 'multidiffusion'] = 'stochastic',
+        max_resolution: int =1024,
         remove_bg: bool = True,
     ) -> dict:
         """
@@ -368,7 +374,7 @@ class TrellisImageTo3DPipeline(Pipeline):
             preprocess_image (bool): Whether to preprocess the image.
         """
         if preprocess_image:
-            images = [self.preprocess_image(image,remove_bg=remove_bg) for image in images]
+            images = [self.preprocess_image(image, max_resolution, remove_bg=remove_bg) for image in images]
         cond = self.get_cond(images)
         cond['neg_cond'] = cond['neg_cond'][:1]
         torch.manual_seed(seed)

utils/depth_estimation.py

@@ -9,7 +9,7 @@ from pathlib import Path
 import logging
 logging.getLogger("transformers.modeling_utils").setLevel(logging.ERROR)
 from utils.image_utils import (
-    resize_image_with_aspect_ratio
+    resize_image_with_aspect_ratio, multiply_and_blend_images, open_image
 )
 from utils.constants import TMPDIR
 from easydict import EasyDict as edict

utils/image_utils.py

@@ -4,6 +4,7 @@ from io import BytesIO
 import cairosvg
 import base64
 import numpy as np
+import rembg
 #from decimal import ROUND_CEILING
 from PIL import Image, ImageChops, ImageDraw, ImageEnhance, ImageFilter, ImageDraw, ImageOps, ImageMath
 from typing import List, Union, is_typeddict
@@ -363,18 +364,15 @@ def resize_image_with_aspect_ratio(image, target_width, target_height):
         # Image is taller than target aspect ratio
         new_height = target_height
         new_width = int(target_height * original_aspect)
-
     # Resize the image
     resized_image = image.resize((new_width, new_height), Image.LANCZOS)
     #print(f"Resized size: {resized_image.size}\n")
-
     # Create a new image with target dimensions and black background
     new_image = Image.new("RGB", (target_width, target_height), (0, 0, 0))
     # Paste the resized image onto the center of the new image
     paste_x = (target_width - new_width) // 2
     paste_y = (target_height - new_height) // 2
     new_image.paste(resized_image, (paste_x, paste_y))
-
     return new_image
 
 def lerp_imagemath(img1, img2, alpha_percent: int = 50):
@@ -463,10 +461,8 @@ def multiply_and_blend_images(base_image, image2, alpha_percent=50):
     base_image = base_image.convert('RGBA')
     image2 = image2.convert('RGBA')
     image2 = image2.resize(base_image.size)
-
     # Multiply the images
     multiplied_image = ImageChops.multiply(base_image, image2)
-
     # Blend the multiplied result with the original
     blended_image = Image.blend(base_image, multiplied_image, alpha)
     if name is not None:
@@ -1369,4 +1365,21 @@ def calculate_optimal_fill_dimensions(image: Image.Image):
     width = max(width, BASE_HEIGHT) if width == FIXED_DIMENSION else width
     height = max(height, BASE_HEIGHT) if height == FIXED_DIMENSION else height
 
-    return width, height
+    return width, height
+
+
+def combine_depth_map_with_image_path(image_path, depth_map_path, output_path, alpha: int= 95) -> str:
+    image =open_image(image_path)
+    depth_map = open_image(depth_map_path)
+    image = image.resize(depth_map.size)
+    depth_map = depth_map.convert("RGBA")
+    depth_no_background = rembg.remove(depth_map, session = rembg.new_session('u2net'))
+    overlay = Image.blend(image,depth_no_background, alpha= (alpha / 100))
+    overlay.save(output_path)
+    return output_path
+
+def combine_depth_map_with_Image(image:Image, depth_img:Image, width:int, height:int, alpha: int= 95, rembg_session_name: str ='u2net') -> Image:
+    resized_depth_image = resize_image_with_aspect_ratio(depth_img, width, height)
+    depth_no_background = rembg.remove(resized_depth_image, session = rembg.new_session(rembg_session_name))
+    combined_depth_img = multiply_and_blend_images(image, depth_no_background, (alpha / 100))
+    return combined_depth_img