Data Fusion | Yonghui Liu

Urban Knowledge and Semantic Intelligence

Sun, 08 Mar 2026 00:00:00 +0000

Motivation

Trajectories describe movement through a city; urban semantics add context to those movements. Road networks sit within rich information about points of interest, land use, functional zones, and road classes. Much of this context exists as unstructured text, or in datasets that use incompatible category systems. Many road graph models emphasize topology and numeric attributes. Integrating heterogeneous semantic context reliably remains a challenge.

This direction combines large language models, which can read and compare unstructured descriptions, with graph learning, which captures network structure, to build road network representations that carry urban semantics and transfer across cities and datasets.

Research Questions

RQ1: Cross-taxonomy alignment. Datasets from different cities and platforms use different category systems, for example for POIs or road classes. How can LLMs help align these labels automatically, with little human annotation?
RQ2: Transferable road representations. How can a semantic graph learning framework learn road network representations that transfer to cities it was not trained on?
RQ3: Selective knowledge integration. Adding large amounts of text-derived context to a road graph can introduce noise or negative transfer. How can a model decide which semantic context is relevant for a given road and task?

Work So Far

RQ1 (Semantic alignment): We combined LLM prompt-based pseudo-labels with DeepWalk structural embeddings to align road labels across taxonomies (OpenStreetMap highway tags to FHWA functional classes). The method reaches 79.1% accuracy versus 54.3% for prompt-only mapping, and degrades far more gracefully than direct supervision when labels are scarce (51.3% vs. 40.0% accuracy with 1% of labels). ( , published in IEEE ITSC 2025)
RQ2 (Transferable representations): We proposed HetSemNet + SaRNSAGE, which fuses LLM-encoded POI context with road topology in a heterogeneous graph. It outperforms GCN, GAT, GraphSAGE, and a heterogeneous graph-attention baseline on road type classification (best F1 0.7296), and a model trained on Porto transfers unchanged to Lisbon with only a 4.71% F1 drop. ( , published in IEEE ITSC 2025)
RQ3 (Selective integration): Ongoing work on gating mechanisms that control which urban semantics are fused into road-centric graph representations. (Under review)

Outlook

Road network representations that carry semantic context could support applications such as LLM-assisted urban planning, semantic map generation for autonomous driving, and context-aware traffic management.

A Generalizable Semantic Graph Learning Framework for Road Network Representation

Tue, 18 Nov 2025 00:00:00 +0000

Road networks are the backbone input for tasks like traffic prediction and autonomous navigation, but representing them well is hard: they are large, their topology keeps changing, and most of the useful context around a road (nearby shops, amenities, land use) exists only as unstructured text rather than a tidy set of features. Graph neural networks such as GCN, GAT, and GraphSAGE improved on hand-crafted road attributes by learning from network topology, but they typically look only at the roads themselves and ignore this surrounding semantic context, or risk adding noise if it is folded in without a principled mechanism.

This paper proposes HetSemNet + SaRNSAGE, a two-part framework for fusing that semantic context with road topology:

HetSemNet turns the road network into a heterogeneous graph with two node types: road nodes (initialized with structural attributes like segment length and speed limit) and context nodes representing nearby points of interest (POIs). Roads connect to each other along real road connectivity, and each road also connects to nearby context nodes within a spatial search radius, with the connection weighted by a distance-based decay so that closer POIs count for more. Context nodes are not linked to one another, which keeps their influence local. Each context node’s text description (e.g., a facility name or category) is turned into a fixed-size vector by a frozen, pre-trained language model, so the graph carries topology and semantics side by side.
SaRNSAGE is a GraphSAGE-style encoder adapted to this richer graph. For each road node, it mean-aggregates information from all of its road neighbors, and separately mean-aggregates information from a fixed number of sampled context neighbors. It also computes a second, simpler signal from that same sampled set of context neighbors: a plain average of their raw, un-aggregated LLM embeddings, so that fine-grained semantic detail isn’t lost when the learned aggregation compresses it. These signals — road, aggregated context, and raw context — are concatenated and passed through a learned update to produce the road’s embedding. The encoder itself is trained self-supervised, without labels: connected road pairs are treated as positive examples and randomly-sampled unconnected pairs as negatives, optimized with a binary cross-entropy loss.

The learned embeddings were evaluated on a road-type classification task, using real urban road-network data from Porto split 7:2:1 into train/validation/test, with the encoder frozen and only a small MLP classifier trained on top for evaluation. Every graph-based baseline beat a handcrafted-feature MLP (F1 = 0.4835), but adding semantic context produced the largest jumps: even a simple mean-pooling of POI embeddings (SaMean) lifted F1 by 29.53% over that baseline, and the full SaRNSAGE model reached the best overall F1 of 0.7296 — a 50.90% improvement over the handcrafted baseline and a 16.50% improvement over SaMean. This beat every other baseline tested, including semantic-augmented versions of GCN, GAT, and GraphSAGE (which, per the paper, still operate on a homogeneous, road-only graph even in this comparison) and SaHAN, a heterogeneous graph-attention baseline that, like SaRNSAGE, runs on the full HetSemNet graph.

Because SaRNSAGE only needs to sample a fixed, small neighborhood around each node, it can produce embeddings for roads and cities it has never seen without retraining. The paper tests this directly with a cross-city transfer experiment: a model trained only on Porto is applied, unchanged, to Lisbon, a city that differs noticeably from Porto in layout, scale, and facility distribution. The transferred model reaches an F1 of 0.6766, compared with 0.7296 in the Porto evaluation and 0.6968 for a model trained directly on Lisbon. These results show that the learned representations transfer to another city without retraining, with some loss in classification performance.

F1-score and training time as a function of the number of sampled context neighbors K.

A separate sweep over the number of sampled context neighbors, K, tells a complementary efficiency story: F1 rises quickly with K and then levels off, while training time keeps climbing steeply as K grows, and using every nearby POI with no sampling at all (“All” in the figure) adds a large amount of extra training time for essentially no further gain in accuracy. This is the practical case for SaRNSAGE’s sampling-and-aggregation design: a moderate, fixed sample size gets most of the benefit of semantic context at a fraction of the computational cost, which matters for deploying the model on large, evolving road networks where full retraining is impractical.

Cross-Taxonomy Label Alignment in Transportation Networks With Prompt-Driven Large Language Models

Tue, 18 Nov 2025 00:00:00 +0000

Transportation datasets rarely speak the same language. A city’s OpenStreetMap extract might tag a road as “primary” or “secondary,” while a government inventory such as the U.S. Federal Highway Administration’s (FHWA) Functional Classification sorts the same roads into categories like “Interstate,” “Principal Arterial,” or “Minor Collector.” Point-of-interest catalogs and land-use datasets run into the same mismatch in their own vocabularies. Merging sources like these for infrastructure planning or traffic analysis means mapping one taxonomy onto another, and today that mostly still means manually built lookup tables or brittle keyword-matching rules — approaches that don’t scale and break down as soon as a new dataset phrases things slightly differently.

This paper proposes a framework that automates that mapping using two complementary signals instead:

Prompt-driven pseudo-labeling. For each source-taxonomy label, a large language model is queried with several different prompt variants asking which target-taxonomy label it corresponds to; the candidate answers are combined by majority vote into one pseudo-label. This gives the model a starting signal without any hand-built mapping table.
Structural embeddings. In parallel, a DeepWalk embedding is trained directly on the road network graph, so every road segment also carries information about where it sits and how it connects in the network, independent of what it happens to be called.

The two views — an entity’s LLM-derived semantic signal (pseudo-label plus its encoded source-taxonomy label) and its structural embedding — are concatenated into one multi-view input and fed to a two-layer classifier.

Training happens in two stages. The model first pretrains on the LLM-generated pseudo-labels, using a technique the authors call semantic masking: in every batch, a random subset of nodes has its semantic input zeroed out, which forces the model to also lean on the structural embedding rather than just memorizing the pseudo-label mapping. It’s then fine-tuned on a small set of human-verified labels to correct whatever noise the pseudo-labels introduced. Tested on mapping real OSM ‘highway’ tags to FHWA functional classes, the fine-tuned model reached 79.1% accuracy and 81.0% macro-F1, well ahead of plain string-matching (29.5%/20.9%) and of using the LLM’s pseudo-labels directly with no learned model at all (54.3%/51.8%). An ablation confirmed both views pull their weight: a model trained on structural embeddings alone already reached 68.5% accuracy, versus 54.8% for a model with no structural information at all — a sizeable share of the framework’s benefit comes from network topology, not just the LLM’s semantic guesses.

The more telling result is what happens as human-verified labels become scarce, which is the realistic case for most cross-dataset integration projects. The paper compares the full framework (pseudo-label pretraining + fine-tuning) against a model trained the conventional way — directly and only on whatever labeled data happens to be available.

Performance as the human-labeled fine-tuning budget shrinks from 100% to 0.1%: the LLM-pretrained-then-fine-tuned model (orange) degrades far more gracefully than a model trained the conventional way, directly supervised on the same shrinking label budget (blue).

With abundant labels the two approaches land close together (both around 79-81%), but as the labeled fraction shrinks, the directly-supervised model’s performance collapses — down to 40.0% accuracy / 33.4% macro-F1 at just 1% of the labels, and below 30% accuracy at 0.1%. The pseudo-label-pretrained model degrades far more gently: still 51.3% accuracy / 50.4% macro-F1 at 1% of the labels, and 47.0% accuracy even at 0.1%. That gap is the practical payoff — real cross-taxonomy alignment problems rarely come with abundant hand-labeled data across every source, and this is exactly where the prompt-driven pretraining earns its keep, giving the model a useful starting point before it ever sees a human annotation.