spotify/scio   0.9.6

A Scala API for Apache Beam and Google Cloud Dataflow.

spotify/ratatool   0.4.10

A tool for data sampling, data generation, and data diffing

spotify/magnolify   0.7.4

A collection of Magnolia add-on modules

googleclouddataproc/spark-bigquery-connector   0.41.0

BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.

nevillelyh/shapeless-datatype   0.2.0

Shapeless utilities for common data types

samelamin/spark-bigquery   0.2.7

Google BigQuery support for Spark, Structured Streaming, SQL, and DataFrames with easy Databricks integration.

starlake-ai/starlake   1.3.0

Declarative text based tool for data analysts and engineers to extract, load, transform and orchestrate their data pipelines.

tharwaninitin/etlflow   1.7.3

EtlFlow is an ecosystem of functional libraries in Scala based on ZIO for running complex Auditable workflows which can interact with Google Cloud Platform, AWS, Kubernetes, Databases, SFTP servers, On-Prem Systems and more.

wix-incubator/quix   1.0.11

Quix Notebook Manager

ocadaruma/scalikejdbc-bigquery   0.1.4

ScalikeJDBC extension for Google BigQuery

cartodb/analytics-toolbox-core   2022.10.19

A set of UDFs and Procedures to extend BigQuery, Snowflake, Redshift, Postgres and Databricks with Spatial Analytics capabilities

data-tools/big-data-types   1.3.4

A library to transform Scala product types and Schemes from different systems into other Schemes. Any implemented type automatically gets methods to convert it into the rest of the types and vice versa. E.g: a Spark Schema can be transformed into a BigQuery table.

spotify/scio-contrib   0.0.1

Community-supported add-ons for Scio

armanbilge/gcp4s   0.1-ff546f8

Cross-platform JVM/JS Google Cloud Platform integrations for fs2 and friends

fpopic/bigquery-schema-select   0.4

(Script) Generates SQL query that selects all fields (recursively for nested fields) from the provided BigQuery schema file.