{"id":744,"date":"2024-04-29T17:36:41","date_gmt":"2024-04-29T17:36:41","guid":{"rendered":"https:\/\/www.volucer.it\/?p=744"},"modified":"2024-04-29T17:39:52","modified_gmt":"2024-04-29T17:39:52","slug":"ai-in-action-basic-concepts-about-llm","status":"publish","type":"post","link":"https:\/\/www.volucer.it\/?p=744","title":{"rendered":"AI in Action: BASIC CONCEPTS ABOUT LLM"},"content":{"rendered":"\n

LLMs - Large Language Models<\/strong><\/p>\n\n\n\n

LLMs are neural networks that can process and generate natural language text.<\/p>\n\n\n

\n
\"Midjourney<\/a>
Midjourney Bot APP \u2014 Today at 19:22 LLMs are neural networks that can process and generate natural language text. - Image #1 @Ahdpea8<\/figcaption><\/figure>\n<\/div>\n\n\n

<\/p>\n\n\n\n

TRAINING PHASE<\/strong><\/p>\n\n\n\n

They are trained on a dataset of billions of sentences using unsupervised learning techniques. In the training process<\/strong> LLMs learn what is the most likely word to came next to the previous one based on huge amount of data.<\/p>\n\n\n

\n
\"\"<\/a><\/figure>\n<\/div>\n\n\n

INPUT BY USER<\/strong><\/p>\n\n\n\n

LLMs accept as input a text prompt<\/strong> by a user and in relation with it generate<\/strong> in output text, word by word (token by token).<\/p>\n\n\n\n

GENERATION OF THE OUTPUT<\/strong><\/p>\n\n\n\n

The generation process<\/strong> consists in predicting the next word on the base of previously generated words. LLMs are trained in doing this without any consciousness which is a prerogative of the human mind.<\/p>\n\n\n\n

<\/p>\n\n\n\n

Building a Simple Large Language Model<\/strong><\/h2>\n\n\n\n

In this example we use as data the dystopian novel \u201cNineteen Eighty-Four \u2013 <\/em>1984\u201d by English writer George Orwell, published on 1949.<\/p>\n\n\n\n

Using the text of the novel as a data source, the following tables were produced. I show only a part of them:<\/p>\n\n\n\n

Word\/Token<\/strong><\/td>Occurrences<\/strong><\/td><\/tr>
the<\/strong><\/td>6249<\/td><\/tr>
of<\/strong><\/td>3309<\/td><\/tr>
a<\/strong><\/td>2482<\/td><\/tr>
and<\/strong><\/td>2326<\/td><\/tr>
to<\/strong><\/td>2236<\/td><\/tr>
was<\/strong><\/td>2213<\/td><\/tr>
He<\/strong><\/td>1959<\/td><\/tr>
It<\/strong><\/td>1864<\/td><\/tr>
in<\/strong><\/td>1759<\/td><\/tr>
that<\/strong><\/td>1457<\/td><\/tr>
had<\/strong><\/td>1311<\/td><\/tr>
his<\/strong><\/td>1079<\/td><\/tr>
you<\/strong><\/td>1011<\/td><\/tr>
not<\/strong><\/td>827<\/td><\/tr>
with<\/strong><\/td>771<\/td><\/tr>
as<\/strong><\/td>672<\/td><\/tr>
At<\/strong><\/td>654<\/td><\/tr>
they<\/strong><\/td>642<\/td><\/tr>
for<\/strong><\/td>615<\/td><\/tr>
IS<\/strong><\/td>614<\/td><\/tr>
but<\/strong><\/td>611<\/td><\/tr>
be<\/strong><\/td>608<\/td><\/tr>
on<\/strong><\/td>604<\/td><\/tr>
were<\/strong><\/td>583<\/td><\/tr>
there<\/strong><\/td>559<\/td><\/tr>
Winston<\/strong><\/td>526<\/td><\/tr>
him<\/strong><\/td>512<\/td><\/tr>
i<\/strong><\/td>495<\/td><\/tr>
which<\/strong><\/td>443<\/td><\/tr>
s<\/strong><\/td>439<\/td><\/tr>
one<\/strong><\/td>426<\/td><\/tr>
or<\/strong><\/td>424<\/td><\/tr>
\u2026<\/td>\u2026<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
Word\/Token<\/strong><\/td>Word Next<\/strong><\/td>Score<\/strong><\/td>Probability<\/strong><\/td><\/tr>
of<\/td>the<\/td>743<\/td>0,01139<\/td><\/tr>
It<\/td>was<\/td>589<\/td>0,00903<\/td><\/tr>
in<\/td>the<\/td>574<\/td>0,00880<\/td><\/tr>
He<\/td>had<\/td>355<\/td>0,00544<\/td><\/tr>
he<\/td>was<\/td>273<\/td>0,00418<\/td><\/tr>
on<\/td>the<\/td>230<\/td>0,00352<\/td><\/tr>
was<\/td>a<\/td>225<\/td>0,00345<\/td><\/tr>
there<\/td>was<\/td>223<\/td>0,00342<\/td><\/tr>
to<\/td>the<\/td>212<\/td>0,00325<\/td><\/tr>
O<\/td>Brien<\/td>205<\/td>0,00314<\/td><\/tr>
to<\/td>be<\/td>203<\/td>0,00311<\/td><\/tr>
and<\/td>the<\/td>203<\/td>0,00311<\/td><\/tr>
had<\/td>been<\/td>202<\/td>0,00310<\/td><\/tr>
the<\/td>party<\/td>195<\/td>0,00299<\/td><\/tr>
at<\/td>the<\/td>183<\/td>0,00280<\/td><\/tr>
that<\/td>he<\/td>167<\/td>0,00256<\/td><\/tr>
from<\/td>the<\/td>161<\/td>0,00247<\/td><\/tr>
with<\/td>a<\/td>158<\/td>0,00242<\/td><\/tr>
did<\/td>not<\/td>148<\/td>0,00227<\/td><\/tr>
that<\/td>the<\/td>147<\/td>0,00225<\/td><\/tr>
of<\/td>a<\/td>145<\/td>0,00222<\/td><\/tr>
of<\/td>his<\/td>145<\/td>0,00222<\/td><\/tr>
out<\/td>of<\/td>142<\/td>0,00218<\/td><\/tr>
was<\/td>not<\/td>130<\/td>0,00199<\/td><\/tr>
with<\/td>the<\/td>127<\/td>0,00195<\/td><\/tr>
he<\/td>could<\/td>124<\/td>0,00190<\/td><\/tr>
it<\/td>is<\/td>124<\/td>0,00190<\/td><\/tr>
in<\/td>his<\/td>123<\/td>0,00188<\/td><\/tr>
in<\/td>a<\/td>122<\/td>0,00187<\/td><\/tr>
They<\/td>were<\/td>122<\/td>0,00187<\/td><\/tr>
seemed<\/td>to<\/td>115<\/td>0,00176<\/td><\/tr>
was<\/td>the<\/td>110<\/td>0,00169<\/td><\/tr>
could<\/td>not<\/td>109<\/td>0,00167<\/td><\/tr>
he<\/td>said<\/td>109<\/td>0,00167<\/td><\/tr>
the<\/td>same<\/td>103<\/td>0,00158<\/td><\/tr>
for<\/td>the<\/td>101<\/td>0,00155<\/td><\/tr>
by<\/td>the<\/td>95<\/td>0,00146<\/td><\/tr>
for<\/td>a<\/td>92<\/td>0,00141<\/td><\/tr>
into<\/td>the<\/td>92<\/td>0,00141<\/td><\/tr>
she<\/td>had<\/td>87<\/td>0,00133<\/td><\/tr>
as<\/td>though<\/td>82<\/td>0,00126<\/td><\/tr>
they<\/td>had<\/td>80<\/td>0,00123<\/td><\/tr>
that<\/td>it<\/td>80<\/td>0,00123<\/td><\/tr>
have<\/td>been<\/td>79<\/td>0,00121<\/td><\/tr>
and<\/td>a<\/td>78<\/td>0,00120<\/td><\/tr>
it<\/td>had<\/td>77<\/td>0,00118<\/td><\/tr>
The<\/td>other<\/td>76<\/td>0,00116<\/td><\/tr>
of<\/td>them<\/td>76<\/td>0,00116<\/td><\/tr>
to<\/td>him<\/td>75<\/td>0,00115<\/td><\/tr>
the<\/td>telescreen<\/td>75<\/td>0,00115<\/td><\/tr>
BIG<\/td>BROTHER<\/td>73<\/td>0,00112<\/td><\/tr>
\u2026<\/td>\u2026<\/td>\u2026<\/td>\u2026<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

This is a simple diagram to understand how the text is generated word by word.<\/p>\n\n\n\n

For example, if I start with BIG, LLM will probably generate BROTHER, and continuing we can produce this sentence:<\/p>\n\n\n\n

BIG<\/strong><\/td>BROTHER<\/strong><\/td>was<\/strong><\/td>a<\/strong><\/td>sort<\/strong><\/td>of<\/strong><\/td>the<\/strong><\/td>thought<\/strong><\/td>\u2026<\/td><\/tr>
Probability<\/td>0,00112<\/td>0,0005<\/td>0,00345<\/td>0,00095<\/td>0,00104<\/td>0,00023<\/td>0,00087<\/td>
<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

<\/p>\n\n\n\n

ChatGPT is a LLM<\/strong><\/h2>\n\n\n\n

By using \u201cprompt<\/strong>\u201d mechanism you can ask ChatGPT for what you want using the natural language.<\/p>\n\n\n\n

But how ChatGPT \u201cUNDERSTAND\u201d text inserted by the user?<\/p>\n\n\n\n

The text is transformed and each word represented by a code that computer can processed.<\/p>\n\n\n\n

A way to represent individual words is Word2Vec<\/strong> technique in natural language processing (NLP), in which each word is represented by a vector (a set of numbers). This helped a computer to assign a meaning to the word.<\/p>\n\n\n\n

Word2Vec<\/strong> stands for \u201cwords as vectors<\/strong>\u201d. It means expressing each word in your text corpus in n-dimensional space. The word\u2019s weight in each dimension defines it for the model.<\/p>\n\n\n\n

The meaning of the words is based on the context defined by its neighboring words where they are associated.<\/p>\n\n\n\n

A simple example of word representation using the Word2Vec approach in two-dimensional space.<\/p>\n\n\n\n

Man = [1,4]<\/p>\n\n\n\n

Woman = [1,3]<\/p>\n\n\n\n

Manager = [4,2]<\/p>\n\n\n\n

Actress = [4,1]<\/p>\n\n\n\n

Manager<\/strong><\/td>-<\/strong><\/td>Man<\/strong><\/td>+<\/strong><\/td>Woman<\/strong><\/td>=<\/strong><\/td>Actress<\/strong><\/td><\/tr>
[4,2]<\/td>[1,4]<\/td>[1,3]<\/td>[4,1]<\/td><\/tr><\/tbody><\/table>


In the following picture we have the graphic representation.
<\/figcaption><\/figure>\n\n\n\n

<\/p>\n\n\n

\n
\"\"<\/a><\/figure>\n<\/div>\n\n\n

This is what happens when you sent some prompt to ChatGPT.<\/p>\n\n\n\n

    \n
  1. The text is converted and split in tokens;<\/li>\n<\/ol>\n\n\n\n

    [10,10], [10,31], [10,15], [14,44], [8,5], \u2026<\/p>\n\n\n\n

    (you, are , an, ICT, specialist, with, a, lot, of, experience)<\/p>\n\n\n\n

      \n
    1. An algorithm (like ChatGPT) makes some prediction and output text word by word.<\/li>\n<\/ol>\n\n\n\n

      [10,10],\u2026<\/p>\n\n\n\n

      (you,can,have, an, important, and, well-paid, job)<\/p>\n\n\n\n

      <\/p>\n\n\n\n

      <\/p>\n\n\n\n

      ChatGPT and PROMPT ENGINEERING<\/strong><\/h2>\n\n\n\n

      Let us now analyze some techniques to better exploit the potential of ChatGPT.<\/p>\n\n\n\n

      DIRECTIONAL PROMPTING<\/strong><\/p>\n\n\n\n

      If you submit the same question to ChatGPT many times, you will likely receive different answers.<\/p>\n\n\n\n

      How can you use directional prompting in order to get more precise answer?<\/p>\n\n\n\n

      You have to give more information and to be more descriptive when you define a prompt. You have to give clear instruction. This will help the model to understand of what you want. If you ask for generic question, you receive generic answer.<\/p>\n\n\n\n

      Generic question:<\/p>\n\n\n

      \n
      \"\"<\/a><\/figure>\n<\/div>\n\n\n

      More specific question:<\/p>\n\n\n

      \n
      \"\"<\/a><\/figure>\n<\/div>\n\n\n

      More contextual and specific question:<\/p>\n\n\n

      \n
      \"\"<\/a><\/figure>\n<\/div>\n\n\n

      <\/p>\n\n\n\n

      <\/p>\n\n\n\n

      OUTPUT FORMATTING<\/strong><\/p>\n\n\n\n

      If you want to have a specific output or format of the output from ChatGPT, for example CSV (Comma Separated Values<\/em>), Microsoft Excel, Microsoft Word or simply txt or maybe code as well, you have to specify as in the following examples.<\/p>\n\n\n\n

      We want statistical data in CSV format:<\/p>\n\n\n

      \n
      \"\"<\/a><\/figure>\n<\/div>\n\n\n

      BIBLIOGRAPHY\/WEBOGRAPHY<\/strong><\/h2>\n\n\n\n

      [01]<\/strong> openai.com;<\/p>\n\n\n\n

      [02]<\/strong> KENNETH WARD CHURCH, Emerging Trends Word2Vec<\/em>, IBM 2016;<\/p>\n\n\n\n

      <\/p>\n\n\n\n

      Not generated by AI tools or platforms.<\/strong><\/p>\n\n\n\n

      {[(homo scripsit)]}<\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"

      LLMs - Large Language Models LLMs are neural networks that can process and generate natural language text. TRAINING PHASE They are trained on a dataset of billions of sentences using unsupervised learning techniques. In the training process LLMs learn what is the most likely word to came next to the previous one based on huge ...continue reading →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[29,30,32],"tags":[],"class_list":["post-744","post","type-post","status-publish","format-standard","hentry","category-artificial-intelligence","category-massive-multimodal-models","category-natural-language-processing-nlp"],"_links":{"self":[{"href":"https:\/\/www.volucer.it\/index.php?rest_route=\/wp\/v2\/posts\/744","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.volucer.it\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.volucer.it\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.volucer.it\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.volucer.it\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=744"}],"version-history":[{"count":22,"href":"https:\/\/www.volucer.it\/index.php?rest_route=\/wp\/v2\/posts\/744\/revisions"}],"predecessor-version":[{"id":773,"href":"https:\/\/www.volucer.it\/index.php?rest_route=\/wp\/v2\/posts\/744\/revisions\/773"}],"wp:attachment":[{"href":"https:\/\/www.volucer.it\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=744"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.volucer.it\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=744"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.volucer.it\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=744"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}