Custom Objectives

Overview link

By default, HpMocd optimizes two built-in objectives implemented in Rust: intra-community density and inter-community separation. These are evaluated in parallel via Rayon and cover the most common use case.

Since v2.0.2, you can replace these defaults with your own Python objective functions via the objectives parameter. This lets you optimize for any community quality metric — modularity density, conductance, motif-based measures, or anything else you can express as a function.

Writing an Objective Function link

Each objective must follow this signature:

  def objective(graph, partition: dict[int, int]) -> float:
    ...
  

Simple Example link

A single objective that maximizes Newman modularity:

  import networkx as nx
from pymocd import HpMocd

def negative_modularity(G, partition):
    communities = {}
    for node, cid in partition.items():
        communities.setdefault(cid, []).append(node)
    return -nx.community.modularity(G, communities.values())

G = nx.karate_club_graph()
alg = HpMocd(G, objectives=[negative_modularity], pop_size=50, num_gens=50)
solution = alg.run()
  

You can pass any number of objectives — the algorithm uses NSGA-II to find Pareto-optimal trade-offs across all of them:

  alg = HpMocd(G, objectives=[obj_1, obj_2, obj_3])
  

Factory Pattern for Performance link

Each objective is called once per individual, per generation — that is pop_size x num_gens times. Recomputing graph properties inside the function body every time is wasteful.

The factory pattern solves this: a function receives the graph once, precomputes constants, and returns a fast closure:

  import numpy as np
import scipy.sparse as sp

def make_conductance(G):
    # Precompute once (called at construction time)
    nodes = list(G.nodes())
    idx = {v: i for i, v in enumerate(nodes)}
    n = len(nodes)
    src = [idx[u] for u, v in G.edges()] + [idx[v] for u, v in G.edges()]
    dst = [idx[v] for u, v in G.edges()] + [idx[u] for u, v in G.edges()]
    A = sp.csr_matrix((np.ones(len(src)), (src, dst)), shape=(n, n))
    degrees = np.asarray(A.sum(axis=1)).ravel()
    total_vol = degrees.sum()
    rows, cols = A.nonzero()

    # Fast closure (called pop_size x num_gens times)
    def _obj(_G, partition):
        if total_vol == 0:
            return 0.0
        labels = np.array([partition[v] for v in nodes], dtype=np.int32)
        n_comms = labels.max() + 1
        is_cut = (labels[rows] != labels[cols]).astype(np.float64)
        cut_comm = np.bincount(labels[rows], weights=is_cut, minlength=n_comms)
        vol_comm = np.bincount(labels, weights=degrees, minlength=n_comms)
        vol_comp = total_vol - vol_comm
        denom = np.minimum(vol_comm, vol_comp)
        mask = denom > 0
        return float((cut_comm[mask] / denom[mask]).mean()) if mask.any() else 0.0

    return _obj
  

Usage — note that the factory is called with G at construction time, and the returned closure is what HpMocd calls repeatedly:

  alg = HpMocd(
    G,
    objectives=[make_conductance(G)],
    pop_size=50,
    num_gens=50,
)
solution = alg.run()
  

Example: Motif-Based Objectives (HPMOCD-II) link

A more advanced example using triangle (3-clique) motifs as the optimization target. Two complementary objectives:

1. Motif Average Degree — maximizes the density of triangles inside each community
2. Motif Conductance — minimizes the fraction of triangles crossing community boundaries

Both use sparse matrix algebra (trace(B^3)/6) to count internal triangles without any Python loops — making them efficient even on large graphs.

  from pymocd import HpMocd
from lfr_experiment.objectives import make_motif_average_degree, make_motif_conductance

alg = HpMocd(
    G,
    objectives=[make_motif_average_degree(G), make_motif_conductance(G)],
    pop_size=50,
    num_gens=50,
)
solution = alg.run()
  

See the full implementation in tests/benchmarks/lfr_experiment/objectives.py.

Changing Objectives After Construction link

Use set_objectives() to swap objectives on an existing instance. Pass an empty list to revert to the built-in Rust objectives:

  alg = HpMocd(G)

# Switch to custom objectives
alg.set_objectives([make_conductance(G)])

# Revert to built-in Rust objectives
alg.set_objectives([])
  

Progress Tracking link

Register a callback with set_on_generation() to monitor the evolutionary process. The callback is invoked after every generation:

  from tqdm import tqdm

alg = HpMocd(G, objectives=[make_conductance(G)], pop_size=50, num_gens=50)

bar = tqdm(total=alg.num_gens, desc="HpMocd", unit="gen")

def on_gen(generation, num_gens, front_size):
    bar.set_postfix(front=front_size)
    bar.update(1)
    if generation == num_gens - 1:
        bar.close()

alg.set_on_generation(on_gen)
solution = alg.run()
  

Performance Considerations link

Recommendations:

• Reduce the evolutionary budget: use pop_size=50, num_gens=50 instead of the default 100/100. The benchmark suite uses these reduced values for all Python-objective variants.
• Precompute graph constants: use the factory pattern to avoid redundant work inside the objective closure.
• Use vectorised operations: leverage numpy and scipy.sparse instead of Python loops. The motif objectives use B @ B (sparse matmul) and np.bincount — all executed in C-level code.
• Profile first: the built-in Rust objectives are 10-30x faster per evaluation. Only use custom objectives when you need a metric that the defaults do not capture.