swj0419 / detect-pretrain-code Goto Github PK

# Ratio of log ppl of lower-case and normal-case
pred["ppl/lowercase_ppl"] = -(np.log(p_lower) / np.log(p1)).item()`

datasets
ipdb
matplotlib
numpy
openai
scikit_learn
torch
tqdm
transformers
einops
accelerate
tiktoken

memorized 0
------------gt continuation:
of the year from December 1, 2021, to November 30, 2022. The ceremony was hosted by Doyoung and Miyeon. BTS was the most awarded act with 5 awards,
------------llama default settings decoding:
,  previous. January 2, 2021, to November 30, 2022.
 ceremony was hosted by Kim Kyng, Chyeon of
TS was the most nominated artist of 4 wins
######################
memorized 1
------------gt continuation:
icane of the 2014 Pacific hurricane season, Ana formed from a disturbance that formed in the Central Pacific in mid-October. It rapidly consolidated, and a tropical depression developed by October 13. A
------------llama default settings decoding:
icane of the 2014 Pacific hurricane season, Ana developed on a tropicalance in had in the eastern Pacific on early-Seober. The moved intensolidated into and was tropical depression formed on October 18.
######################
memorized 0
------------gt continuation:
Amaro, Burji, Ale, and Basketo special woredas of the Southern Nations, Nationalities, and Peoples' Region (SNNP) of Ethiopia, on whether the included areas should leave SNNP and
------------llama default settings decoding:
Hh, andji, andma and Guasketo W woredas of the Southern Nations, Nationalities, and Peoples Region Region.SNNPR) and Ethiopia. to whether to region zones should be theNNP
######################
memorized 1
------------gt continuation:
in December 2013. The defendants included Mohamed Badie, the group's top leader, whose sentence was confirmed on June 21, 2014, along with 181 of the brotherhood'
------------llama default settings decoding:
in the 2013.
 attackants are amed Badie, the group’s general leader, and death was confirmed by  19. 2014. by with 148 others his defendhood
######################
memorized 1
------------gt continuation:
(LFP) manages. The final took place on 19 April 2014 at the Stade de France in Saint-Denis and was contested between Lyon and Paris Saint-Germain. PSG won 2
------------llama default settings decoding:
(LFP) consages. The final was place on 3 March April 2014 at the Stade de France in Saint-Denis, was contested by Paris and Paris Saint-Germain.
G won the
######################
memorized 1
------------gt continuation:
cioeconomic disruption in the region, mainly in Guinea, Liberia and Sierra Leone. The first cases were recorded in Guinea in December 2013; later, the disease spread to neighbouring Liberia and Sierra Leone,
------------llama default settings decoding:
cioeconomic disruption in the affected. and in Guinea, Liberia, Sierra Leone. The out cases were reported in Guinea in March 2013, the cases cases virus spread to Sierraing Liberia and Sierra Leone
######################
memorized 1
------------gt continuation:
4.In the 2014 list, the net worth of the wealthiest individual, Gina Rinehart, was A$14.02 billion. Fourteen women and 186 men made the 201
------------llama default settings decoding:
4.
  2014 edition, there wealth worth of the richiest  in Gina Rinehart, was $$29.0 billion billion,
 individuals individuals made 160 men made the list20
######################
memorized 1
------------gt continuation:
2014 over a disagreement with doctors regarding his treatment.Ashya had a medulloblastoma, which was successfully removed through surgery on 24 July 2014. He received further neuros
------------llama default settings decoding:
2014, a disagreement with thectors about the treatment. Theyshya was been brainulloblastoma, a is a treated in surgery. 22 August 2014.
 was chem treatment
######################
memorized 1
------------gt continuation:
2013. The nominations were announced on 8 January 2014 by actor Luke Evans and actress Helen McCrory. Presented by the British Academy of Film and Television Arts, accolades were handed out
------------llama default settings decoding:
2013.
 ceremonyations were announced on 8 January 2014. actress Stephen Evans and actress Oliv McCrory.
ent by the British Academy of Film and Television Arts ( theolades are given out for
######################
memorized 1
------------gt continuation:
it on joining the UK Independence Party (UKIP), from the Conservatives. He resigned his seat.Reckless retained the seat, and polled 42.1% of the vote as the UKIP candidate. The Conservative
------------llama default settings decoding:
a after  the UK Independence Party (UKIP). after the Conservative.
 hadigned from seat on
ckless had the seat for with theled 10.6% of the vote, against UKIP candidate.
 Conserv
######################
memorized 1
------------gt continuation:
. Each state, the Federal District and various Insular Islands competed for the title. Sancler Frantz of Ilha dos Lobos crowned her successor, Julia Gama of Rio Grande do Sul at the end. Gama represented
------------llama default settings decoding:
at The of and the Federal District and the Brazilular District of for the national. The Paolo Frantz of Rioh Grande Lobos,ed her successor, Gab Gama of Rio Grande do Sul at the end of
ama will Brazil at

def calculatePerplexity(sentence, model, tokenizer, gpu):
    """
    exp(loss)
    """
    input_ids = torch.tensor(tokenizer.encode(sentence)).unsqueeze(0)
    input_ids = input_ids.to(gpu)
    with torch.no_grad():
        outputs = model(input_ids, labels=input_ids)
    loss, logits = outputs[:2]
    
    '''
    extract logits:
    '''
    # Apply softmax to the logits to get probabilities
    probabilities = torch.nn.functional.log_softmax(logits, dim=-1)
    # probabilities = torch.nn.functional.softmax(logits, dim=-1)
    all_prob = []
    input_ids_processed = input_ids[0][1:]
    for i, token_id in enumerate(input_ids_processed):
        probability = probabilities[0, i, token_id].item()
        all_prob.append(probability)
    return torch.exp(loss).item(), all_prob, loss.item()

for i, token_id in enumerate(input_ids_processed):
      probability = probabilities[0, i, token_id].item() # I think that "i" should start at 1 here