Comments (4)
👍
from scio.
http://blog.jetbrains.com/scala/2015/10/14/intellij-api-to-build-scala-macros-support/
It's not as straightforward as I thought since one will have to implement a custom injector like this:
https://github.com/JetBrains/intellij-scala/blob/idea16.x/src/org/jetbrains/plugins/scala/lang/psi/impl/toplevel/typedef/simulacrum/SimulacrumInjection.scala
from scio.
Right now the closest workaround, with #111, would be:
Make a query in the REPL, fetch result schema and get case class definitions with SchemaUtil.toPrettyString()
.
scio> val query = "SELECT repository.* FROM [bigquery-public-data:samples.github_nested] LIMIT 1000"
query: String = SELECT repository.* FROM [bigquery-public-data:samples.github_nested] LIMIT 1000
scio> val schema = bq.getTableSchema(bq.query(query))
...
scio> com.spotify.scio.bigquery.types.SchemaUtil.toPrettyString(schema, "Row", 0)
res1: String =
@BigQueryType.toTable
case class Row(repository: Option[Repository$])
case class Repository$(url: Option[String], has_downloads: Option[Boolean], created_at: Option[String], has_issues: Option[Boolean], description: Option[String], forks: Option[Int], fork: Option[Boolean], has_wiki: Option[Boolean], homepage: Option[String], integrate_branch: Option[String], master_branch: Option[String], size: Option[Int], private: Option[Boolean], name: Option[String], organization: Option[String], owner: Option[String], open_issues: Option[Int], watchers: Option[Int], pushed_at: Option[String], language: Option[String])
And then copy these case class definitions to the job code and use sc.typedBigQuery[Row](query)
.
from scio.
Documented in https://github.com/spotify/scio/wiki/BigQueryType-and-IntelliJ-IDEA
Doesn't look like it'll be fixed in IntelliJ any time soon so closing this for now.
from scio.
Related Issues (20)
- Taps with PubsubIO HOT 2
- SmbIO doesn't work when SortedBucketTransforms have SideInputs
- IntelliJ complains about lack of implicit Coders (bug introduced in Scio 0.14) HOT 1
- Add option for SMB joins to fall back to regular joins
- SMB writes with CharSequence key can't be read with
- TransformOverride#ofSource fails with empty Seq input
- distinctBy and distinctByKey trigger warning about combineByKey HOT 1
- Add a utility helper for testing Parquet Projections/FilterPredicates HOT 1
- Improve error message when sparkey hits array-size limits
- Better projection support for Parquet SMB reads
- AvroCoder fails on nullable Decimal types w/ Avro 1.11.3 HOT 1
- Improve memory usage in scio-extra's sortValues API
- Allow largeHash* and sparkey methods to set a byte size target
- Integration with OpenTelemetry HOT 4
- typedSparkeySideInput cannot load null values into cache HOT 1
- Label SMB Reads with metadata about SortedBucketIO class
- Importing protobuf Any breaks coder derivation
- Benchmark Vectored IO support for scio-parquet/scio-smb HOT 1
- Support for Kafka HOT 1
- Add Errorhandling to BigTableIO HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from scio.