Exporting your data in one huge file is often a big problem for the next steps in your workflow. Terality now offers the possibility to export a dataframe into multiple files, with two extra methods to the pandas API: to_csv_folder and to_parquet_folder.
The number of files can be specified in several ways:: the number of files, the number of rows per file, or the in-memory size per file.
More information on our documentation: https://docs.terality.com/getting-terality/api-reference/write-to-multiple-files.
Terality user dashboard:
Added support for Apple Silicon Macs
To check how Terality compares to the best solutions on the market, we picked the most scientific, unbiased and well-known benchmark for pandas alternatives: the h2o benchmark. It consists of a list of timed simulations on different database-like operations like: join, merge, and groupby, run on different dataset sizes: 0.5, 5 and 50GB. You can check the final section where we give more detail on the experiments and how to reproduce the results for Terality.
After weeks of preparation, we’re proud to finally announce Terality hosted demo notebook - the fastest way to take Terality for a test ride, completely free of charge. We wanted to lower the time needed for you to realize what Terality is all about to 1 click! There’s no better way than running a pre-written tutorial on our infrastructure to experience our pandas lightning-fast serverless data processing