Basics: loading CDMs

Basics: loading CDMs#

import kessler

Load CDMs from .kvn#

In this tutorial, we show how to load CDMs from .kvn format.

First, the CDMs in .kvn format need to be placed inside the path_to_cdms_folder, for correctly loading the data. Furthermore, the code expects the CDMs in the folder to have file names grouped by: individual event and the CDM sequence in each event.

For instance, if we have to load two events with 3 and 2 CDMs each, we might then have file names in the following format:

  • event_1_01.cdm.kvn.txt

  • event_1_02.cdm.kvn.txt

  • event_1_03.cdm.kvn.txt

  • event_2_01.cdm.kvn.txt

  • event_2_02.cdm.kvn.txt

from kessler import EventDataset

We can then proceed in creating the EventDataset object:

path_to_cdms_folder='cdm_data/cdms_kvn/'

events=EventDataset(path_to_cdms_folder)
#A message appears confirming that the loading has happened, with the number of CDMs and events.

Loading CDMS (with extension .cdm.kvn.txt) from directory: /Users/giacomoacciarini/cdm_data/cdms_kvn/
Loaded 39 CDMs grouped into 4 events

Loading CDMs from pandas DataFrame object#

In this tutorial, we show how to load CDMs from pandas DataFrame object.

First we perform the relevant imports:

import kessler
import pandas as pd
from kessler import EventDataset

Then, we create the EventDataset object, after having uploaded the pandas dataframe and created the DataFrame object:

file_name='cdm_data/cdms_csv/sample.csv'
df=pd.read_csv(file_name)
events=EventDataset.from_pandas(df)

Dataframe with 2 rows and 231 columns
Dropping columns with NaNs
Dataframe with 2 rows and 104 columns
Grouping by event_id
Grouped into 1 event(s)
Converting DataFrame to EventDataset
Time spent  | Time remain.| Progress             | Events | Events/sec
0d:00:00:00 | 0d:00:00:00 | #################### | 1/1 | 404.06         

EventDataset(Events:1, number of CDMs per event: 2 (min), 2 (max), 2.00 (mean))

Loading CDMs from Kelvins Challenge dataset#

In this tutorial, we show the case in which the data to be loaded comes from the Kelvins competition: a collision avoidance challenge organized by ESA in 2019.

For this purpose, we built a specific converter that takes care of the conversion from the Kelvins format to standard CDM format. First, we perform the relevant imports:

import kessler
from kessler.data import kelvins_to_event_dataset

Cannot import dbm.gnu: No module named '_gdbm'

/Users/giacomoacciarini/miniconda3/envs/fdl/lib/python3.7/site-packages/pyprob/util.py:327: UserWarning: Empirical distributions on disk may perform slow because GNU DBM is not available. Please install and configure gdbm library for Python for better speed.
  warnings.warn('Empirical distributions on disk may perform slow because GNU DBM is not available. Please install and configure gdbm library for Python for better speed.')

Then, we proceed in converting the Kelvins dataset as an EventDataset objetc. In the following example, we leverage two extra entries (i.e., drop_features and num_events) to exclude certain features when importing, and to only import a limited number of events (in this case 1000).

file_name='cdm_data/kelvins_data/test_data.csv'
events=kelvins_to_event_dataset(file_name, drop_features=['c_rcs_estimate', 't_rcs_estimate'], num_events=1000)
#The output will show the number of CDMs and events loaded, as they progress.

Loading Kelvins dataset from file name: /Users/giacomoacciarini/cdm_data/kelvins_data/test_data.csv
24484 entries
Dropping features: ['c_rcs_estimate', 't_rcs_estimate']
Dropping rows with NaNs
21932 entries
Removing outliers
19531 entries
Shuffling
Grouped rows into 1726 events
Taking TCA as current time: 2022-02-17 23:50:10.189235
Converting Kelvins challenge data to EventDataset
Time spent  | Time remain.| Progress             | Events    | Events/sec
0d:00:00:07 | 0d:00:00:00 | #################### | 1000/1000 | 128.69
Contents