# Larger Sum(Logits) != Larger Sum(Probability)

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flarger-sum-logits-larger-sum-probability%2F22358%3E%5C
**Category:** 🤗Transformers\
**Created:** [August 31, 2022, 3:07pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flarger-sum-logits-larger-sum-probability%2F22358 "2022-08-31T15:07:02Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Hannibal046](https://avatars.discourse-cdn.com/v4/letter/h/ccd318/32.png) [@Hannibal046](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FHannibal046)%5C
**Post date:** [August 31, 2022, 3:07pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flarger-sum-logits-larger-sum-probability%2F22358%2F1 "2022-08-31T15:07:02Z")

</div>

Hi, I find an interesting problem in `beam_search`. In short, the sequential order by `beam_search` is determined by avg of logits( let’s say length is the same for all candidates). However, the larger avg of logits won’t guarantee the larger avg probability. I am not sure if it is correct.

 ![2611661962244_.pic](https://us1.discourse-cdn.com/hellohellohello/original/2X/9/9676daaecb6ba71cee7f47807b9d786014f03100.jpeg)  
 ![2621661962307_.pic](https://us1.discourse-cdn.com/hellohellohello/original/2X/0/0cd3664135fdfa06393cf35123c01e17acee0e83.jpeg)

Code to reproduce:

```python
# %%
from transformers import BartTokenizer,BartForConditionalGeneration
model_path = "/data/pretrained_model/bart_base"
toker = BartTokenizer.from_pretrained(model_path)
model = BartForConditionalGeneration.from_pretrained(model_path)

# %%
input_tokens = ["what do you think it ? huggingface is a great library. And I enjoy it very much",
                "transformers is so good"]
batch_size = 2
num_beams = 10
max_length = 5
num_return_sequences = 5

# %%
input_ids = toker(input_tokens,return_tensors='pt',padding=True).input_ids
output = model.generate(input_ids,max_length=max_length,num_beams=num_beams,num_return_sequences=num_return_sequences,
                        return_dict_in_generate=True,output_scores=True)

# %%
def get_logits_and_probs(output,num_return_sequence,batch_size,eos_token_id):
    """
    using for-loop to get positional-wise logits and probability
    """
    import torch
    total = num_return_sequence * batch_size
    token_logits = [[] for _ in range(total)]
    token_probs = [[] for _ in range(total)]
    continue_or_not = [True for _ in range(total)]
    for time_step in range(len(output.scores)):
        cur_scores = output.scores[time_step] ## num_beam,vocab_size
        for idx in range(total):
            cur_beam = output.beam_indices[idx][time_step]
            cur_token = output.sequences[idx][time_step+1] ## decoder_start_token_id
            if continue_or_not[idx]:
                token_probs[idx].append(torch.softmax(cur_scores[cur_beam],dim=-1)[cur_token].item())
                token_logits[idx].append(cur_scores[cur_beam][cur_token].item())
            if cur_token==eos_token_id:
                continue_or_not[idx]=False
    return token_logits,token_probs
token_logits,token_probs = get_logits_and_probs(output,num_return_sequences,batch_size,toker.eos_token_id)

# %%
def avg(ls):
    return sum(ls)/len(ls)

## check if my get_logits_and_probs function is correct by compare it with output.sequences_scores
for idx in range(num_return_sequences*batch_size):
    if idx == num_return_sequences:
        print("*"*20)
    print(avg(token_logits[idx]),output.sequences_scores[idx].item())
print("probability")
for idx in range(num_return_sequences*batch_size):
    if idx == num_return_sequences:
        print("*"*20)
    print(avg(token_probs[idx]))

```

---

<div class="post-metadata">

**Author:** ![Hannibal046](https://avatars.discourse-cdn.com/v4/letter/h/ccd318/32.png) [@Hannibal046](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FHannibal046)%5C
**Post date:** [August 31, 2022, 3:11pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flarger-sum-logits-larger-sum-probability%2F22358%2F2 "2022-08-31T15:11:55Z")

</div>

@patrickvonplaten , hi could you please check this out ?

---

<div class="post-metadata">

**Author:** ![joaogante](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/joaogante/32/20106_2.png) [@joaogante](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjoaogante)%5C
**Post date:** [September 9, 2022, 12:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flarger-sum-logits-larger-sum-probability%2F22358%2F3 "2022-09-09T12:56:26Z")

</div>

(see related issue, which contains an answer: [Larger Logits != Larger Probability · Issue #18889 · huggingface/transformers · GitHub](https://github.com/huggingface/transformers/issues/18889))
