alpindale / misc-scripts Goto Github PK
View Code? Open in Web Editor NEWMiscellaneous scripts for various stuff.
License: GNU General Public License v3.0
Miscellaneous scripts for various stuff.
License: GNU General Public License v3.0
Hey, I've noticed that whenever I try to use GPTQ quantized models, it doesn't work. I've noticed it sees it as AWQ and not GPTQ, I don't know why that is because it worked fine yesterday. I tried to use "TheBloke/Toppy-M-7B-GPTQ", but it gives me this before it shuts down the notebook.
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.10/dist-packages/aphrodite/endpoints/kobold/api_server.py", line 334, in <module>
engine = AsyncAphrodite.from_engine_args(engine_args)
File "/usr/local/lib/python3.10/dist-packages/aphrodite/engine/async_aphrodite.py", line 486, in from_engine_args
engine = cls(parallel_config.worker_use_ray,
File "/usr/local/lib/python3.10/dist-packages/aphrodite/engine/async_aphrodite.py", line 270, in __init__
self.engine = self._init_engine(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/aphrodite/engine/async_aphrodite.py", line 306, in _init_engine
return engine_class(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/aphrodite/engine/aphrodite_engine.py", line 108, in __init__
self._init_workers(distributed_init_method)
File "/usr/local/lib/python3.10/dist-packages/aphrodite/engine/aphrodite_engine.py", line 144, in _init_workers
self._run_workers(
File "/usr/local/lib/python3.10/dist-packages/aphrodite/engine/aphrodite_engine.py", line 747, in _run_workers
self._run_workers_in_batch(workers, method, *args, **kwargs))
File "/usr/local/lib/python3.10/dist-packages/aphrodite/engine/aphrodite_engine.py", line 720, in _run_workers_in_batch
output = executor(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/aphrodite/task_handler/worker.py", line 72, in load_model
self.model_runner.load_model()
File "/usr/local/lib/python3.10/dist-packages/aphrodite/task_handler/model_runner.py", line 36, in load_model
self.model = get_model(self.model_config)
File "/usr/local/lib/python3.10/dist-packages/aphrodite/modeling/loader.py", line 76, in get_model
quant_config = get_quant_config(model_config.quantization,
File "/usr/local/lib/python3.10/dist-packages/aphrodite/modeling/hf_downloader.py", line 94, in get_quant_config
return quant_cls.from_config(hf_quant_config)
File "/usr/local/lib/python3.10/dist-packages/aphrodite/modeling/layers/quantization/awq.py", line 67, in from_config
zero_point = cls.get_from_keys(config, ["zero_point"])
File "/usr/local/lib/python3.10/dist-packages/aphrodite/modeling/layers/quantization/base_config.py", line 50, in get_from_keys
raise ValueError(f"Cannot find any of {keys} in the model's "
ValueError: Cannot find any of ['zero_point'] in the model's quantization config
Hey, idk what wrong but whenever I try and let the engine run and generate, at the end I get this instead.
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.