AI Journey

ComfyUi //

a small research study

In this set of images, I explore and experiment with LoRAs and ControlNet parameters in ComfyUI to understand and assemble the right pipeline. Flux.1-dev is used as the primary generative model. Image generation is guided by prompts and constrained by reference images. Most of the time, control is achieved with a combination of canny and depth ControlNets. To increase quality and detail, a tiled ControlNet workflow is used for upscaling with an SDXL model.

The main goal is to produce architectural visualizations by placing cubes—drawn as sketches or detailed 3D models —within specific environments. However, the expected creativity from AI workflow is twofold. First, a more controlled creativity: transforming the cube control images into detailed architectural forms while preserving their shapes. Second, a less controlled creativity: designing an atmosphere and environment appropriate to those forms. The tension between consistency and creativity is treated as a research problem, and explored through series of experimentation.

However, although ComfyUI parameters are the primary focus of this experimentation, the form of control images is equally crucial. Sometimes the process begins and ends with simple, background-free cube images; sometimes, to gain more control over the background, these simple images are collaged with previously generated outputs; and sometimes the control images are prepared with a complete background from the start. Besides this, during this experimentation, the parameters of the composition of the cubes in the given images—such as size, repetition, and spacing—are also taken into account when testing consistency and creativity in image generation.

Moreover, low-resolution or low-consistency generations that suggest interesting environmental ideas are also repurposed as control images and then refined and upscaled for more detail.

here some results…