Comments (9)
@nirmdesai Pages created via MkDocs need manual fixing of links with relative paths. We are aware of this and I have asked Shivdeep to do this.
from data-prep-kit.
And to be clear, these links work when viewing from github.com, just seems mkdocs is doing something wrong. For example, the .py links work as expected from https://github.com/IBM/data-prep-lab/blob/dev/data-processing-lib/doc/transform-tutorials.md
from data-prep-kit.
@shivdeep-singh-ibm Have you looked at this and found no solution yet? If there is no solution for referring to python pages in transforming repo with MkDoc to Pages, we should link to Readme pages in the respective directories. As for the formatting issue (the third link above), I think there should be a way to fix this, no?
from data-prep-kit.
I have found 1 way. I am preparing a patch for it. That method is working for python cases, trying to handle some corner cases as well.
The approach is to use hooks.py as a hook to mkdocs , which will automatically update the links (relative links to python files or relative links to repo folders), with. absolute github links to repo on the fly while generatig the documentation.
eg.
[transform](./transform/src/main.py)
will become [transform](https://github.com/IBM/data-prep-lab/blob/dev/transform/src/main.py)
this way it will open github on clicking the link.
I need to support only
- relative links to python files
- relative links to folders
- not to update http/ssh/protocl type links
from data-prep-kit.
Sounds good, @shivdeep-singh-ibm! Thank you.
from data-prep-kit.
I see that the link to the python files has been fixed, but the formatting issue in the page https://ibm.github.io/data-prep-kit/data-processing-lib/doc/architecture/ is still there.
from data-prep-kit.
@shivdeep-singh-ibm Thanks for making the file a lot better by adding new lines. Sorry for nitpicking, but there is still a problem with the indention of bullets and sub-bullets, as I compare the repo Readme with the corresponding Pages version of the architecture.md file. As I look at the markdown file, I see a different color of * for bullets and sub-bullets. Repo treats this correctly, but Pages doesn't. I think the sub-bullets that are red color * should become black for the pages to work properly.
from data-prep-kit.
@shivdeep-singh-ibm @shahrokhDaijavad Is this done? Can it be closed?
from data-prep-kit.
@Bytes-Explorer and @shivdeep-singh-ibm . This is mostly done. The problem is still with the indentation of sub-bullets in this page: https://ibm.github.io/data-prep-kit/data-processing-lib/doc/architecture/) (compare with this page: https://github.com/IBM/data-prep-kit/blob/dev/data-processing-lib/doc/architecture.md in which sub-bullets in the Ray Orchestrator section and Data Access under core components are not indented correctly. I don't know if there is a solution for this (maybe adding a return after the corresponding lines in the md file?) It is not a big issue, if there is no solution and we can close it.
from data-prep-kit.
Related Issues (20)
- Running fdedup in the Notebook examples directory has a bug HOT 2
- [Feature] pyarrow parquet write_table can save up to 30% storage with compression flag βZSTDβ HOT 1
- [Feature] Enable an embeddable mode
- [Bug] Add transform to example notebook in context of Issue#283
- [Bug] Update documentation of repo level ordering transform
- [Bug] Add tests for repo level ordering module
- Improve ray store used in repo level ordering module.
- [Bug] Add kfp support in context of Issue#283
- [Bug] get_config_parameter returns without checking if the config value exists
- [Logging Feature] Logging INFO about completed x files in y mins should add (xx1 successfully and xx2 failed)
- [Bug] Resize behaves badly when there are lots of schema changes
- Tokenizer transform logs are filled with docs info and chuck index when parameter tkn_chunk_size is specified
- Add repo_name column to code2parquet tranform
- [Feature] Enable transform() to terminate all processing of documents across all instances
- [Feature] Capability to chunk text for RAG systems
- [Feature] Create vector embeddings
- Demo of data-prep-kit for RAG
- [Bug] Failing to publish repo_level_ordering
- [Feature] Create a demo notebook for RAG
- [Bug] pdf2parquet test failure when running locally (passes in github worflow)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from data-prep-kit.