Skip to content

BigQuerySource does not validate whether the table exists or not #1596

@achals

Description

@achals

Expected Behavior

When specifying an invalid table, feast should surface an error that explicitly states this.

Current Behavior

We currently get an error that implies that the schema cannot be parsed.

$ feast apply
Traceback (most recent call last):
  File "/Users/achal/tecton/feast/.direnv/python-3.7.10/bin/feast", line 33, in <module>
    sys.exit(load_entry_point('feast', 'console_scripts', 'feast')())
  File "/Users/achal/tecton/feast/.direnv/python-3.7.10/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/achal/tecton/feast/.direnv/python-3.7.10/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/achal/tecton/feast/.direnv/python-3.7.10/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/achal/tecton/feast/.direnv/python-3.7.10/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/achal/tecton/feast/.direnv/python-3.7.10/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/achal/tecton/feast/.direnv/python-3.7.10/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/achal/tecton/feast/sdk/python/feast/cli.py", line 192, in apply_total_command
    apply_total(repo_config, repo)
  File "/Users/achal/tecton/feast/sdk/python/feast/telemetry.py", line 151, in exception_logging_wrapper
    result = func(*args, **kwargs)
  File "/Users/achal/tecton/feast/sdk/python/feast/repo_operations.py", line 131, in apply_total
    repo = parse_repo(repo_path)
  File "/Users/achal/tecton/feast/sdk/python/feast/repo_operations.py", line 97, in parse_repo
    module = importlib.import_module(module_path)
  File "/usr/local/opt/python@3.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/achal/tecton/feast/splendid_lark/example.py", line 46, in <module>
    table_ref="feast-oss.demo_data.driver_activity"
  File "/Users/achal/tecton/feast/sdk/python/feast/data_source.py", line 677, in __init__
    or self._infer_event_timestamp_column("TIMESTAMP|DATETIME"),
  File "/Users/achal/tecton/feast/sdk/python/feast/data_source.py", line 547, in _infer_event_timestamp_column
    """
TypeError:
                    Unable to infer DataSource event_timestamp_column due to an absence of columns that satisfy the criteria.
                     Please specify event_timestamp_column explicitly.

Steps to reproduce

  • Create a feature view with an invalid BigQuery source.

driver_stats_fv = FeatureView(
    name="driver_activity",
    entities=["driver_id"],
    input=BigQuerySource(
        table_ref="feast-oss.demo_data.driver_activity"
    )
)

and then run feast apply

Specifications

  • Version:
  • Platform:
  • Subsystem:

Possible Solution

Run client.get_table as validation before any other operations.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions