artemis.generators.csvgen

Generator for csv-like data

Module Contents

class artemis.generators.csvgen.GenCsvLike

Creates data in CSV format and sends bytes. Generator tries to guess at payload size for total number of floats to generate / column / chunk

gen_chunk(self, ncolumn, unit, size)

Create a chunk of data of ncolumns, of size <size> in <unit>.

generate(self)

class artemis.generators.csvgen.GenCsvLikeArrowOptions

nbatches = 1

nsamples = 1

num_cols = 2

num_rows = 10

linesep =

header = True

class artemis.generators.csvgen.GenCsvLikeArrow(name, **kwargs)

Bases: artemis.generators.common.GeneratorBase

Arrow-like generator see arrow/python/pyarrow/tests/test_csv.py

tests specific number of rows and columns sends a batch rather than Table

pa_types = ['int32', 'uint32', 'int64', 'uint64', 'float32', 'float64']

property num_batches(self)

static generate_col_names()

initialize(self)

make_random_csv(self)

make_mixed_random_csv(self)

Use arrow commons builtins to generate arrow arrays and push to csv precursor to using adds

generate(self)

sampler(self)

__next__(self)

write(self)