You might be interested in this workshop paper I worked on, exploring improvements &am

Hi <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

Workshop paper on improvements & limitations to MADDNESS about halutmatmul HOT 2 CLOSED

joennlae commented on May 31, 2024

Workshop paper on improvements & limitations to MADDNESS

from halutmatmul.

Comments (2)

joennlae commented on May 31, 2024 1

Hi @calvinmccarter

Thank you for the suggestion. I read it and agree with your assessment, especially with: "We suggest that future research focus on improving the MADDNESS hash function, which appears to be an accuracy bottleneck.". If you want to use more data to back up that statement, feel free to use my results in addition to yours.

I am thinking about the LeViT results:

Explanation of the table.

MADDness: Is the in the paper described encoding algorithm.
DT: Using the same hashed binary decision tree, the learning is top-down instead of bottom-up. First building the centroids and then mapping the hash function with a decision tree classifier.

halutmatmul/src/python/halutmatmul/decision_tree_and_pq.py

Lines 201 to 207 in d48ac55

 centroids_, assignments_ = kmeans2(X, K, minit="points", iter=5) 

 end = timer() 

 print(f"kmeans time {end - start}") 

 # kmeans = KMeans(n_clusters=K, n_init=1).fit(X) 

 # kmeans = BisectingKMeans(n_clusters=K, n_init=1).fit(X) 

 # centroids_, assignments_ = kmeans.cluster_centers_, kmeans.labels_ 

 clf_ = tree.DecisionTreeClassifier(**decision_tree_args)

PQ: Product quantization: mapping the input vector to the closest prototype. So it is essentially the perfect encoding function.

Knowing this we can say pretty exact how much we could gain with a better encoding function or in your words: how big the accuracy bottleneck is :-).

ResNet-50 results show the accuracy bottleneck.

I wish you a good presentation :-)

from halutmatmul.

calvinmccarter commented on May 31, 2024

@joennlae, thanks -- this analysis is so helpful!

from halutmatmul.

Workshop paper on improvements & limitations to MADDNESS about halutmatmul HOT 2 CLOSED

Comments (2)

Related Issues (1)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

	centroids_, assignments_ = kmeans2(X, K, minit="points", iter=5)
	end = timer()
	print(f"kmeans time {end - start}")
	# kmeans = KMeans(n_clusters=K, n_init=1).fit(X)
	# kmeans = BisectingKMeans(n_clusters=K, n_init=1).fit(X)
	# centroids_, assignments_ = kmeans.cluster_centers_, kmeans.labels_
	clf_ = tree.DecisionTreeClassifier(**decision_tree_args)