I would like to implement a function that can generate AST expressions, for example, b

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

FWIW we'd also be OK with changing the current model, see <a class="issue-link js-issu

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

[QST] Recusively Generating AST Expressions (C++ libcudf) about cudf HOT 8 OPEN

shriramch commented on May 30, 2024

[QST] Recusively Generating AST Expressions (C++ libcudf)

from cudf.

Comments (8)

bdice commented on May 30, 2024

@shriramch You can use an owning object like a std::deque<expression> or std::list<expression> where references are stable (std::vector can reallocate and invalidate references). The lifetime of the AST expressions would then be tied to the lifetime of that container, and you can return it or pass it around.

from cudf.

bdice commented on May 30, 2024

For an example of constructing an expression tree, see this benchmark:

cudf/cpp/benchmarks/ast/transform.cpp

Lines 65 to 87 in 23bb2ed

 // Note that a std::list is required here because of its guarantees against reference invalidation 

 // when items are added or removed. References to items in a std::vector are not safe if the 

 // vector must re-allocate. 

 auto expressions = std::list<cudf::ast::operation>(); 

 // Construct tree that chains additions like (((a + b) + c) + d) 

 auto const op = cudf::ast::ast_operator::ADD; 

 if (reuse_columns) { 

 expressions.push_back(cudf::ast::operation(op, column_refs.at(0), column_refs.at(0))); 

 for (cudf::size_type i = 0; i < tree_levels - 1; i++) { 

 expressions.push_back(cudf::ast::operation(op, expressions.back(), column_refs.at(0))); 

 } 

 } else { 

 expressions.push_back(cudf::ast::operation(op, column_refs.at(0), column_refs.at(1))); 

 std::transform(std::next(column_refs.cbegin(), 2), 

 column_refs.cend(), 

 std::back_inserter(expressions), 

 [&](auto const& column_ref) { 

 return cudf::ast::operation(op, expressions.back(), column_ref); 

 }); 

 } 

 auto const& expression_tree_root = expressions.back();

from cudf.

vyasr commented on May 30, 2024

FWIW we'd also be OK with changing the current model, see #10744. If you would like to make that change, a PR is welcome!

from cudf.

shriramch commented on May 30, 2024

@bdice Thank you, this approach worked.

from cudf.

shriramch commented on May 30, 2024

@vyasr Sure, I'd like to try this.

from cudf.

vyasr commented on May 30, 2024

@shriramch OK great! Feel free to post here if you need help in getting a development environment set up for cudf!

from cudf.

shriramch commented on May 30, 2024

@vyasr Is it possible to get a remote development environment? Unfortunately I can't do this on my current PC.

from cudf.

vyasr commented on May 30, 2024

Would you be able to spin up an EC2 or GCP instance for this? I think getting access on one of the cloud services would be the easiest. Paperspace is another good option.

from cudf.

Recommend Projects

[QST] Recusively Generating AST Expressions (C++ libcudf) about cudf HOT 8 OPEN

Comments (8)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

	// Note that a std::list is required here because of its guarantees against reference invalidation
	// when items are added or removed. References to items in a std::vector are not safe if the
	// vector must re-allocate.
	auto expressions = std::list<cudf::ast::operation>();

	// Construct tree that chains additions like (((a + b) + c) + d)
	auto const op = cudf::ast::ast_operator::ADD;
	if (reuse_columns) {
	expressions.push_back(cudf::ast::operation(op, column_refs.at(0), column_refs.at(0)));
	for (cudf::size_type i = 0; i < tree_levels - 1; i++) {
	expressions.push_back(cudf::ast::operation(op, expressions.back(), column_refs.at(0)));
	}
	} else {
	expressions.push_back(cudf::ast::operation(op, column_refs.at(0), column_refs.at(1)));
	std::transform(std::next(column_refs.cbegin(), 2),
	column_refs.cend(),
	std::back_inserter(expressions),
	[&](auto const& column_ref) {
	return cudf::ast::operation(op, expressions.back(), column_ref);
	});
	}

	auto const& expression_tree_root = expressions.back();