This time, I decided to take on an extreme challenge to push both my fundamental machine learning skills and my engineering intuition to their limits.

The subject was CubiCasa5K, one of the most well-known datasets and the de facto standard for floor plan analysis.

GitHub - CubiCasa/CubiCasa5k: CubiCasa5k floor plan dataset

CubiCasa5k floor plan dataset. Contribute to CubiCasa/CubiCasa5k development by creating an account on GitHub.

GitHubCubiCasa

No Papers. No GitHub. Just the Data.

Normally, when working with a famous dataset, the first step is obvious: read the official paper, clone the official GitHub repository, and build upon existing work.

This time, I deliberately prohibited myself from doing any of that.

I gave myself only one rule:

No papers. No source code. Using only the floor plan images and their SVG annotations, reverse-engineer an entire floor plan parsing system completely from scratch.

You might wonder why anyone would intentionally make things harder for a technology that was published seven years ago.

Once I actually started, however, the experience turned out to be one of the most exciting and educational projects of my career. Without documentation, specifications, or reference implementations, I had no choice but to "talk" directly to the data itself. What followed was a long series of trial and error—but also one of the most rewarding learning experiences I've ever had.

What is CubiCasa5K?

CubiCasa5K is a large-scale dataset with dense annotations designed for automatic floor plan analysis and digitization (vectorization).

Its greatest strength lies in pairing real-world, low-quality raster floor plan images with fully structured vector-format ground truth data.

The dataset contains approximately 5,000 floor plans collected from real estate documents. Because these images originate from scans and photocopies, they naturally include many real-world imperfections such as faded lines, distortion, skew, and low resolution.

Meanwhile, the ground truth precisely labels architectural elements—including walls, windows, doors, and room categories—using vector polygons.

Because the task requires reconstructing clean, structured information from noisy real-world inputs, CubiCasa5K has become one of the most widely used benchmarks for developing and evaluating robust machine learning models.

Choosing the Benchmark Floor Plan

The floor plan below was selected as the benchmark for this experiment.

At the top of the drawing, two symmetrical apartment units appear to be present, although the image itself has been roughly cropped.

The most interesting part is the small room labeled "S" in the upper-right corner, located beyond the toilet and shower room. This room—and the adjacent space—is most likely a sauna.

Why can we make that assumption?

Because the drawing comes from a Finnish residential property. CubiCasa5K is built from Finnish real estate data, and in Finland it is very common for private apartments and houses to include a personal sauna located next to the bathroom.

This creates an interesting challenge for AI.

Unlike ordinary doors, the sauna door is not represented by a rectangular door symbol. Instead, the model has to infer its existence solely from the swing arc indicating the opening direction.

Preparing the Ground Truth

As shown earlier, every scanned floor plan in CubiCasa5K comes with an accurately reconstructed SVG drawing.

However, these SVG files cannot be used directly as training targets for the CNN in this project.

Therefore, I converted the vector annotations into semantic segmentation masks by assigning a different color to each architectural element.

• Red: Walls
• Blue: Windows
• Green: Doors
• Gray: Floor
• White: Vertices

Training Begins

Training was configured to run for up to 100 epochs, with early stopping once validation accuracy stopped improving.

As expected, a completely untrained model produced meaningless predictions.

In the earliest outputs, the network appeared to discover that painting large areas red resulted in a relatively small penalty. However, it still viewed walls as thin lines rather than solid regions, so it merely traced their outlines.

The model also had no understanding of the blue class. Instead of detecting windows, it colored almost every non-wall region blue. Since blue specifically represents windows, this resulted in a large penalty.

The First Epoch

By the end of the first epoch, the predictions had improved dramatically.

The model had already undergone an important conceptual shift—it no longer treated walls as simple lines, but as complete surfaces.

Even more surprisingly, it had begun to ignore neighboring rooms instead of allowing predictions to bleed across room boundaries.

Although windows and doors were still somewhat unreliable, the network had already learned an important structural rule:

Windows and doors almost always appear along walls.

A Simple Trick That Greatly Accelerated Learning

One major reason the model learned so quickly was careful data preparation.

During training, I randomly rotated every floor plan by different angles.

Without this augmentation, the model would only ever see perfectly aligned floor plans, causing it to understand the world only within a neat horizontal and vertical grid.

In such an "easy mode" environment—where walls, doors, and windows are always parallel or perpendicular—the network quickly finds shortcuts instead of learning the true geometric relationships.

By constantly exposing the model to randomly rotated floor plans, it was forced to develop a deeper understanding of each architectural component while remaining challenged throughout training.

Random rotation also helps reduce overfitting.

In simple terms, overfitting means memorizing the training data instead of learning general patterns.

Even with approximately 5,000 floor plans, repeatedly training on the same images can eventually cause the model to memorize them.

Random rotations effectively create countless new variations of the existing data, increasing the apparent size of the dataset and improving the model's ability to generalize to unseen floor plans.

Successfully Detecting Doors

As training progressed, the model gradually began recognizing windows and doors.

The most interesting case is the sauna door in the upper-right corner of the floor plan. Unlike the other doors, this one has no rectangular door outline, making it considerably more difficult to detect. Even so, the model was already showing early signs of recognizing it.

This indicates that the AI was not simply learning the rule that "doors are always rectangles." Instead, it had begun to understand that the door swing arc is also an important visual cue indicating the presence of a door. Without learning this relationship, it would have been impossible to detect the sauna door.

The fact that this behavior emerged as early as Epoch 2 was an extremely encouraging sign.

Another notable improvement is that, by this point, the model had completely stopped being influenced by neighboring rooms.

Improving Recognition Accuracy

After Epoch 2, training continued by refining the patterns the model had already learned.

The predicted boundaries became progressively cleaner, sharper, and more geometrically consistent.

Eventually, the sauna door was detected just as accurately as every other door in the floor plan.

The only remaining issue was the floor region in what appears to be the balcony at the upper-left corner. This area contains several complex drawing elements that do not belong to any of the semantic classes defined in this experiment, so the misclassification is understandable.

Since the overall segmentation quality was already more than sufficient for validation, I decided to stop training at this point.

Automatically Generating a 3D Model

Using the final segmentation result, I generated a 3D model automatically.

After correcting the geometric distortion in the original floor plan, I simply assigned predefined heights to each semantic category based on a set of fixed rules.

As a result, the heights of doors and windows may not exactly match those in the real building.

Nevertheless, compared with the original hand-drawn floor plan, the generated model provides a much more intuitive understanding of the building's overall structure.

Conclusion

This CubiCasa5K project was carried out under a self-imposed reverse-engineering challenge: no official paper, no reference implementation, and no GitHub repository.

Starting from complete uncertainty, I ultimately reached not only high-quality semantic segmentation but also automatic 3D model generation.

Looking back, the biggest reason this experiment succeeded was not simply because I used a modern neural network architecture.

Instead, the key was spending a tremendous amount of effort understanding the architectural context hidden behind the dataset and embedding that domain knowledge into every stage of the pipeline.

In machine learning, simply cloning a powerful library and running it often works only until the model encounters the messy, noisy data of the real world.

To build practical AI systems, engineers need to ask deeper questions:

• Why is the model making mistakes here?
• What assumptions were made when the dataset was created?
• What knowledge about the problem domain is still missing from the model?

Only by continuously having a dialogue with the data and translating domain knowledge into preprocessing strategies, data augmentation, and loss functions can we build AI systems that are truly robust in real-world applications.

This experiment reinforced my belief that this seemingly old-fashioned, trial-and-error approach remains one of the most effective paths toward practical machine learning.

If you have a favorite dataset, I highly recommend trying the same challenge yourself:

No papers. No cloned repositories. Just you and the data.

You may be surprised at how much clearer your understanding becomes.