DataLoaderΒΆ

class dgl.graphbolt.DataLoader(datapipe, num_workers=0, persistent_workers=True)[source]ΒΆ

Bases: Generic[torch.utils.data.dataloader.T_co]

Multiprocessing DataLoader.

Iterates over the data pipeline with everything before feature fetching (i.e. dgl.graphbolt.FeatureFetcher) in subprocesses, and everything after feature fetching in the main process. The datapipe is modified in-place as a result.

Only works on single GPU.

Parameters

  • datapipe (DataPipe) – The data pipeline.

  • num_workers (int, optional) – Number of worker processes. Default is 0.

  • persistent_workers (bool, optional) – If True, the data loader will not shut down the worker processes after a dataset has been consumed once. This allows to maintain the workers instances alive.