The diconet from alexnowakvila

Questions about logp calculation and GNN input in kmeans.py

Hi!

Thank you for your kindness to share your code! Divide and conquer strategy is really awesome.

I have read your paper and Kmeans code carefully with several unclear questions remained. Looking forward to your replay:

logp calculation

As shown below, when last ==false, variable Lgp will be reassigned to a zero tensor at every iteration of k, thus only the last Lgp is calculated in the backward process. Why not take all split steps into account?

DiCoNet/src/Kmeans/kmeans.py

Lines 279 to 296 in 15e48e3

 for k in range(K): 

 scores,_ = gnn(input) 

 probs = F.sigmoid(scores) 

 if mode == 'train': 

 variance = compute_variance(e, probs) 

 variance = variance.sum() / bs 

 Lgp = Variable(torch.zeros(n_samples, bs).type(dtype)) 

 Reward2 = Variable(torch.zeros(n_samples, bs).type(dtype)) 

 Reward3 = Variable(torch.zeros(n_samples, bs).type(dtype)) 

 for i in range(n_samples): 

 Samplei, Lgp[i] = sample_one(probs, 'train') 

 Ei = e*2 + Samplei.long() 

 Reward2[i], _,_ = compute_reward(Ei, k+1, points) 

 baseline = Reward2.mean(0,True).expand_as(Reward3) 

 loss = 0.0 

 if (last and k == K-1) or not last: 

 loss = ((Reward2-baseline) * Lgp).sum(1).sum(0) / n_samples / bs 

 loss_total = loss_total + loss - reg_factor*variance

OP x Y

I find it really hard to understand each variable's functionality, maybe it's because I am not familiar with GNN. Could you simply explain or refer me to some related links? For example, what does W D U in OP means?

DiCoNet/src/Kmeans/kmeans.py

Lines 147 to 163 in 15e48e3

 def create_input(points, sigma2): 

 bs, N, _ = points.size() #points has size bs,N,2 

 OP = torch.zeros(bs,N,N,4).type(dtype) 

 E = torch.eye(N).type(dtype).unsqueeze(0).expand(bs,N,N) 

 OP[:,:,:,0] = E 

 W = points.unsqueeze(1).expand(bs,N,N,dim) - points.unsqueeze(2).expand(bs,N,N,dim) 

 dists2 = (W * W).sum(3) 

 dists = torch.sqrt(dists2) 

 W = torch.exp(-dists2 / sigma2) 

 OP[:,:,:,1] = W 

 D = E * W.sum(2,True).expand(bs,N,N) 

 OP[:,:,:,2] = D 

 U = (torch.ones(N,N).type(dtype)/N).unsqueeze(0).expand(bs,N,N) 

 OP[:,:,:,3] = U 

 OP = Variable(OP) 

 x = Variable(points) 

 Y = Variable(W.clone())

Thanks a lot:)

alexnowakvila / diconet Goto Github PK

diconet's People

Contributors

Stargazers

Watchers

Forkers

diconet's Issues

Questions about logp calculation and GNN input in kmeans.py

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

	for k in range(K):
	scores,_ = gnn(input)
	probs = F.sigmoid(scores)
	if mode == 'train':
	variance = compute_variance(e, probs)
	variance = variance.sum() / bs
	Lgp = Variable(torch.zeros(n_samples, bs).type(dtype))
	Reward2 = Variable(torch.zeros(n_samples, bs).type(dtype))
	Reward3 = Variable(torch.zeros(n_samples, bs).type(dtype))
	for i in range(n_samples):
	Samplei, Lgp[i] = sample_one(probs, 'train')
	Ei = e*2 + Samplei.long()
	Reward2[i], _,_ = compute_reward(Ei, k+1, points)
	baseline = Reward2.mean(0,True).expand_as(Reward3)
	loss = 0.0
	if (last and k == K-1) or not last:
	loss = ((Reward2-baseline) * Lgp).sum(1).sum(0) / n_samples / bs
	loss_total = loss_total + loss - reg_factor*variance

	def create_input(points, sigma2):
	bs, N, _ = points.size() #points has size bs,N,2
	OP = torch.zeros(bs,N,N,4).type(dtype)
	E = torch.eye(N).type(dtype).unsqueeze(0).expand(bs,N,N)
	OP[:,:,:,0] = E
	W = points.unsqueeze(1).expand(bs,N,N,dim) - points.unsqueeze(2).expand(bs,N,N,dim)
	dists2 = (W * W).sum(3)
	dists = torch.sqrt(dists2)
	W = torch.exp(-dists2 / sigma2)
	OP[:,:,:,1] = W
	D = E * W.sum(2,True).expand(bs,N,N)
	OP[:,:,:,2] = D
	U = (torch.ones(N,N).type(dtype)/N).unsqueeze(0).expand(bs,N,N)
	OP[:,:,:,3] = U
	OP = Variable(OP)
	x = Variable(points)
	Y = Variable(W.clone())