jebish7/bge_MNSR

SentenceTransformer based on BAAI/bge-small-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-small-en-v1.5 on the csv dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-small-en-v1.5
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 384 tokens
• Similarity Function: Cosine Similarity
• Training Dataset:
  • csv

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("jebish7/bge_MNSR")
# Run inference
sentences = [
    'How does ADGM ensure that FinTech Participants remain compliant with evolving regulatory standards, particularly in the context of new and developing technologies?',
    'The Guidance is applicable to the following Persons:\n(a)\tan applicant for a Financial Services Permission to carry on the Regulated Activity of Developing Financial Technology Services within the RegLab in or from ADGM; and/or\n(b)\ta FinTech Participant.',
    'DIGITAL SECURITIES – INTERMEDIARIES\nConventional Intermediaries – Digital Securities\nIntermediaries intending to operate solely, in the context of Digital Securities, as a broker or dealer for Clients (including the operation of an OTC broking or dealing desk) are not permitted to structure their broking / dealing service or platform in such a way that would have it be considered as operating a RIE or MTF.  The FSRA would consider features such as allowing for price discovery, displaying a public trading order book (accessible to any member of the public, regardless of whether they are Clients), and allowing trades to automatically be matched using an exchange-type matching engine as characteristic of a RIE or MTF, and not activities acceptable for an Digital Securities intermediary to undertake.\n',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

csv

• Dataset: csv
• Size: 29,545 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 19 tokens mean: 34.47 tokens max: 70 tokens	min: 23 tokens mean: 113.88 tokens max: 512 tokens

• Samples:

  anchor	positive
  In the case of a cross-border transaction involving jurisdictions with differing sanctions regimes, how should a Relevant Person prioritize and reconcile these requirements?	Sanctions. UNSC Sanctions and Sanctions issued or administered by the U.A.E., including Targeted Financial Sanctions, apply in the ADGM. Relevant Persons must comply with Targeted Financial Sanctions. Sanctions compliance is emphasised by specific obligations contained in the AML Rulebook requiring Relevant Persons to establish and maintain effective systems and controls to comply with applicable Sanctions, including in particular Targeted Financial Sanctions, as set out in Chapter ‎11.
  How does the FSRA monitor and assess the deployment scalability of a FinTech proposal within the UAE and ADGM beyond the RegLab validity period?	Evaluation Criteria. To qualify for authorisation under the RegLab framework, the applicant must demonstrate how it satisfies the following evaluation criteria: (a) the FinTech Proposal promotes FinTech innovation, in terms of the business application and deployment model of the technology. (b) the FinTech Proposal has the potential to: i. promote significant growth, efficiency or competition in the financial sector; ii. promote better risk management solutions and regulatory outcomes for the financial industry; or iii. improve the choices and welfare of clients. (c) the FinTech Proposal is at a sufficiently advanced stage of development to mount a live test. (d) the FinTech Proposal can be deployed in the ADGM and the UAE on a broader scale or contribute to the development of ADGM as a financial centre, and, if so, how the applicant intends to do so on completion of the validity period.
  How does the ADGM define "distinct risks" that arise from conducting business entirely in an NFTF manner compared to a mix of face-to-face and NFTF interactions, and what specific risk mitigation strategies should be employed in these scenarios?	The risk assessment under Rule ‎6.2.1(c) should identify actions to mitigate risks associated with undertaking NFTF business generally, and the use of eKYC specifically. This is because distinct risks are often likely to arise where business is conducted entirely in an NFTF manner, compared to when the business relationship includes a mix of face-to-face and NFTF interactions. The assessment should make reference to risk mitigation measures recommended by the Regulator, a competent authority of the U.A.E., FATF, and other relevant bodies.

• Loss: MultipleNegativesSymmetricRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Evaluation Dataset

csv

• Dataset: csv
• Size: 3,676 evaluation samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 18 tokens mean: 34.98 tokens max: 63 tokens	min: 3 tokens mean: 114.8 tokens max: 512 tokens

• Samples:

  anchor	positive
  How should our firm approach the development and implementation of a risk management system that addresses the full spectrum of risks listed, including technology, compliance, and legal risks?	Management of particular risks Without prejudice to the generality of Rule ‎2.4(1, a Captive Insurer must develop, implement and maintain a risk management system to identify and address risks, including but not limited to: (a) reserving risk; (b) investment risk (including risks associated with the use of Derivatives); (c) underwriting risk; (d) market risk; (e) liquidity management risk; (f) credit quality risk; (g) fraud and other fiduciary risks; (h) compliance risk; (i) outsourcing risk; and (j) reinsurance risk. Reinsurance risk refers to risks associated with the Captive Insurer's use of reinsurance arrangements as Cedant.
  What measures could an Authorised Person take to ensure non-repudiation and accountability, so that individuals or systems processing information cannot deny their actions?	In establishing its systems and controls to address information security risks, an Authorised Person should have regard to: a. confidentiality: information should be accessible only to Persons or systems with appropriate authority, which may require firewalls within a system, as well as entry restrictions; b. the risk of loss or theft of customer data; c. integrity: safeguarding the accuracy and completeness of information and its processing; d. non repudiation and accountability: ensuring that the Person or system that processed the information cannot deny their actions; and e. internal security: including premises security, staff vetting; access rights and portable media, staff internet and email access, encryption, safe disposal of customer data, and training and awareness.
  What authority does the Regulator have over the terms and conditions applied to the escrow account holding funds from a Prospectus Offer?	The Regulator may, during the Offer Period or such other longer period as specified, impose a requirement that the monies held by a Person making a Prospectus Offer or his agent pursuant to the Prospectus Offer or issuance are held in an escrow account for a specified period and on specified terms.

• Loss: MultipleNegativesSymmetricRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 64
• learning_rate: 2e-05
• num_train_epochs: 10
• warmup_ratio: 0.1
• load_best_model_at_end: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 10
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	loss
0.8658	200	1.6059	-
1.2684	293	-	0.4773
1.4632	400	0.8247	-
2.2684	586	-	0.4313
2.0606	600	0.7352	-
2.9264	800	1.0011	-
3.2684	879	-	0.4038
3.5238	1000	0.646	-
4.2684	1172	-	0.3926
4.1212	1200	0.6207	-
4.9870	1400	0.8652	-
5.2684	1465	-	0.3769
5.5844	1600	0.5708	-
6.2684	1758	-	0.3691
6.1818	1800	0.5588	-
7.0476	2000	0.7551	-
7.2684	2051	-	0.3608
7.6450	2200	0.5758	-
8.1212	2310	-	0.3561

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.14
• Sentence Transformers: 3.1.1
• Transformers: 4.45.2
• PyTorch: 2.4.0
• Accelerate: 0.34.2
• Datasets: 3.0.1
• Tokenizers: 0.20.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}