Repo for Hamel's Professional Website
hamelsmu / hamel-site Goto Github PK
View Code? Open in Web Editor NEWRepo for Hamel's Professional Website
Home Page: http://hamel.dev/
Repo for Hamel's Professional Website
Home Page: http://hamel.dev/
Repo for Hamel's Professional Website
Hello - thank you for releasing this repo - your website looks really cool!
it'd help me tremendously if you could share some instructions on how I could clone this repo and edit it to create and host a similar website - sorry if you've already shared this or if it's a very basic request
thank you again
Hi Hamel, you must have heard of llama.cpp, I saw your benchmarks on 03_inference.ipynb, but I couldn't see any mention of llama.cpp there. I believe it can run on the same GPU. I don't have that fancy GPU like that so I can't readily benchmark in the same way. TheBloke has this format: https://huggingface.co/TheBloke/Llama-2-7B-GGUF
Maybe you didn't consider it the same class of tool? But it can run a server also, including a OAI style HTTP API.
I found these benchmarks, which show MLC ahead of Llama.cpp, but I wonder if they had been setup to use GPU correctly. It might be worthwhile comparing to latest.
mlc-ai/mlc-llm#15 (comment)
https://github.com/mlc-ai/llm-perf-bench
Anyway thanks for your analysis, pivotal stuff!
@hamelsmu
hello Hamel
hope you don't mind if I ask if you can explain your blogging process which seems to use netlify?
I am a huge fan of you work!
the below function parse_instruction_fields
generates a tuple (instruction,input,response)
The key to making a new input format is to make sure you can parse your input into these parts and deal with them.
Note that the instruction
and input
are part of the user_prompt
which is effectively the the "inputs" for the purposes of train_on_inputs: false
which works by setting the appropriate labels to a label id to be ignored in
tokenized_prompt["labels"] = [IGNORE_INDEX] * user_prompt_len
prompt_tokenizers.InstructionPromptTokenizingStrategy
class InstructionPromptTokenizingStrategy(PromptTokenizingStrategy):
"""
Tokenizing strategy for instruction-based prompts.
"""
def parse_instruction_fields(
self, prompt
) -> Union[Tuple[str, str, str], Tuple[str, str, str, str]]:
raise NotImplementedError
def tokenize_prompt(self, prompt):
(
instruction,
input, # pylint: disable=redefined-builtin
response,
) = self.parse_instruction_fields(prompt)
user_prompt = next(
iter(
self.prompter.build_prompt(
instruction,
input,
)
)
)
tokenized_prompt = self._tokenize(user_prompt, add_eos_token=False)
if not self.train_on_inputs:
user_prompt_len = len(tokenized_prompt["input_ids"])
# TODO this could be sped up using numpy array slicing
tokenized_prompt["labels"] = [IGNORE_INDEX] * user_prompt_len
tokenized_res_prompt = self._tokenize(
response, strip_bos_token=True, add_eos_token=True
)
tokenized_prompt["input_ids"] += tokenized_res_prompt["input_ids"]
tokenized_prompt["attention_mask"] += tokenized_res_prompt["attention_mask"]
tokenized_prompt["labels"] += tokenized_res_prompt["input_ids"]
In Axolotl, many PromptStrategies override the parse_instruct_fields
method like this:
prompt_tokenizers.AlpacaPromptTokenizingStrategy
class AlpacaPromptTokenizingStrategy(InstructionPromptTokenizingStrategy):
"""
Tokenizing strategy for Alpaca prompts.
"""
def parse_instruction_fields(self, prompt) -> Tuple[str, str, str]:
return (
prompt["instruction"],
prompt["input"] if "input" in prompt else "",
prompt["output"],
)
The above is called from prompt_strategies.alpaca_chat
def load(tokenizer, cfg, ds_cfg: Optional[Dict[str, Any]] = None):
prompt_style = PromptStyle.CHAT.value
if ds_cfg and "conversation" in ds_cfg:
prompt_style = ds_cfg["conversation"]
return AlpacaPromptTokenizingStrategy(
AlpacaPrompter(prompt_style),
tokenizer,
cfg.train_on_inputs,
cfg.sequence_len,
)
Why I decided not to commercialize nbdev.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.