Kedro is an open source Python library that helps you build production-ready data and analytics pipelines
Questions tagged [kedro]
90 questions
1
vote
1 answer
Specify host and port in mlflow.yml and run "kedro mlflow ui", but host and port still default (localhost:5000) not change
I build sample kedro project refer to this page,
and specify host as my global ip address in mlflow.yml.
but when I hit "kedro mlflow ui" command, it still listen to local.
even I only specify port to 5001 (not default) in mlflow.yml, it does not…
RCheng
- 11
- 1
1
vote
2 answers
Kedro : Failed to find the pipeline named '__default__'
Having issues with kedro. The 'register_pipelines' function doesn't seem to be running or creating the default Pipeline that I'm returning from it.
The error is
(kedro-environment) C:\Users\cc667216\OneDrive\DCS_Pipeline\dcs_files>kedro…
Cazforshort
- 65
- 7
1
vote
1 answer
kedro run as a python command instead of command line
I am getting started with Kedro, so I created the new kedro project for default iris dataset.
I am able to succesfully run it with kedro run command. My question now is how do I run it as a python command? From the documentation I read that the…
BlueMango
- 375
- 5
- 15
1
vote
1 answer
How do I add xlsb files to the catalog in Kedro?
1.I am using this code in catalog.yml file
equipment_data:
type: pandas.ExcelDataSet
filepath: data\01_raw\Equipment Profile.xlsb
layer: raw
getting error after executing kedro run command.
`
kedro.io.core.DataSetError: Failed while…
Akshay Salvi
- 131
- 5
1
vote
1 answer
How to load a specific catalog dataset instance in kedro 0.17.0?
We were using kedro version 0.15.8 and we were loading one specific item from the catalog this way:
from kedro.context import load_context
get_context().catalog.datasets.__dict__[key]
Now, we are changing to kedro 0.17.0 and trying to load the…
Javi Hernandez
- 107
- 1
- 12
1
vote
2 answers
How do I add a directory of .wav files to the Kedro data catalogue?
This is my first time trying to use the Kedro package.
I have a list of .wav files in an s3 bucket, and I'm keen to know how I can have them available within the Kedro data catalog.
Any thoughts?
Myccha
- 605
- 4
- 15
1
vote
1 answer
Adding pandas dependencies after kedro new
I began a new project with kedro new without adding the files from the iris example. The original requirements.txt looked like:
black==v19.10b0
flake8>=3.7.9, <4.0
ipython~=7.0
isort>=4.3.21, <5.0
jupyter~=1.0
jupyter_client>=5.1, <…
Guilherme
- 23
- 3
1
vote
1 answer
Why my Kedro logging file keeps empty? Am I missing any step?
I am using Kedro but I can't get my logging file to be used. I am following the tutorial. The log file was created but is still empty.
Steps done:
Configured logging
class ProjectContext(KedroContext):
def _setup_logging(self) -> None:
…
Antunes
- 21
- 4
1
vote
2 answers
Can Kedro Create Circular Layers
I am trying to add layer attributes to my catalog. One common pattern I have is to get some data(raw), clean it up, then output a list of parts(pri). I then need metadata for those parts in which I take the list of parts from pri and pass into a…
Waylon Walker
- 443
- 3
- 10
1
vote
2 answers
Read from memory for full pipeline, read from files if retry or partial pipeline
How can I use the pipeline to run from memory/file? I think the features are there but I am not sure how I can write the pipeline like this.
My use case is:
normal pipeline, from step 1 to step 10
run from step 2 to step 10
Imagine at step 1, I…
mediumnok
- 101
- 1
- 6
1
vote
1 answer
Dynamic instance of pipeline execution based on dataset partition/iterator logic
Not sure if this is possible or not, but this is what I am trying to do: -
I want to extract out portions (steps) of a function as individual nodes (ok so far), but the catch is I have an iterator on top of steps, which is dependent on some logic on…
Mohit
- 985
- 3
- 16
- 40
1
vote
1 answer
How to load kedro DataSet object dynamically
I am currently using the yaml api to create all of my datasets with kedro==15.5. I would like to be able to peer into this information from time to time dynamically. It appears that I can get to this information with the io.datasets which is a…
Waylon Walker
- 443
- 3
- 10
1
vote
2 answers
How to run parts of your Kedro pipeline conditionally?
I have a big pipeline, taking a few hours to run. A small part of it needs to run quite often, how do I run it without triggering the entire pipeline?
idanov
- 56
- 5
1
vote
1 answer
How to disable logs in Kedro
I have been unsuccessful in disabling kedro logs. I have tried adding disable_existing_loggers: True to the logging.yml file as well as disable:True to all of the existing logs and it still appears to be saving log files. Any suggestions?
Waylon Walker
- 443
- 3
- 10
0
votes
0 answers
ScannerError: while scanning for the next token
ScannerError: while scanning for the next token
found character that cannot start any token
in "", line 63, column 1
PAXI
- 1