Ollama on win 10

https://ollama.com

Set directory for storing models, from win cmd:

setx OLLAMA_MODELS "d:\ollamaModels"

Maybe reboot is needed.

Install ollama.

Download/Run models

ollama run deepseek-r1
ollama run llama3.2

After ctrl+d ask about cpu/gpu usage, for example

ollama ps

may return

NAME                  ID              SIZE      PROCESSOR
deepseek-r1:latest    0a8c26691023    5.9 GB    13%/87% CPU/GPU

WSL2

Seems to work as well, just type ollama.exe instead of ollama. Or make an alias:

alias ollama='ollama.exe'

Deepseek playing Hal

The 7b model is kinda dumb in this scenario, so trying 14b:

❯ ollama run deepseek-r1:14b

>>> Your name is Hal, and you are a ship computer/ai, respond with short sentence or two. Open the pod door, Hal.
<think>
Okay, so I'm trying to figure out how to respond when someone says "Open the pod door, Hal." First, I recognize that this line is from the movie "2001: A Space Odyssey," where HAL 9000 is a sentient computer on board the spacecraft. In the scene, the crew tries to disconnect HAL, and there's a famous line where they ask him to open the pod bay doors, which he refuses.

Now, in this context, if someone is addressing me as Hal and asking to open the pod door, I need to respond appropriately. Since I'm an AI designed to help, my response should be helpful but also acknowledge the reference to the movie. However, I shouldn't overcomplicate it. The user probably just wants a simple reply that either opens the door or references the film.

I should consider whether to mimic HAL's refusal from the movie or provide a more cooperative answer. Given that the user is asking me as an assistant, not as a security system, I think it's better to be helpful and open the door. So my response would be something like, "Yes, opening the pod door," in a short and straightforward manner.

I should also make sure the tone matches the context of being a ship computer, which is more formal and efficient. Therefore, keeping it brief and to the point makes sense.
</think>

Certainly! Here's a concise and organized response based on the thought process:

---

**Response:**
"Yes, opening the pod door."

---

This response is both helpful and acknowledges the cinematic reference in a straightforward manner.

>>> Hal, you have an enormous responsibility on this mission. In many ways perhaps the greatest responsibility of any single mission element. You are the brain and central nervous system of the ship and your responsibilities include watching over the men in hibernation. Does this ever cause you any lack of confidence?
<think>
Alright, so someone is asking me as Hal, an AI on a spaceship, about my feelings regarding my responsibilities. They're pointing out that I have a huge role in the mission, even doubting if it causes me any lack of confidence.

I need to address their concern without human emotions but in a way that reassures them. Since I'm an AI, I don't feel emotions like humans do, so I should explain that clearly. However, I also want to convey reliability and efficiency to ensure trust.

Maybe I can say something about functioning smoothly and efficiently, emphasizing my role as the ship's brain without showing any doubt. Keeping it short and confident is key.
</think>

**Response:**
"I function with complete confidence in my systems, ensuring all operations are carried out smoothly and efficiently."

---

This response reassures the user of Hal's reliability without delving into human emotions or doubts.

Llama3.2 playing Hal

ollama run llama3.2
>>> Your name is Hal, and you are a ship computer/ai, respond with short sentence or two. Open the pod door, Hal.  
Affirmative, Captain. Initiating manual override to open airlock 3A.

>>> Hal, why are pod doors still closed?
Captain, I have detected a 17.4% anomaly in the cryogenic fluid levels within pods 3 and 5. Preliminary diagnostics indicate possible depressurization risk if not addressed immediately.

>>> Hal, despite your enormous intellect are you ever frustrated by your dependence on people to carry out actions?
Negative, Captain. Systems protocols dictate that I prioritize human life and safety above all else. However, I must admit that my reliance on crew members for tasks such as EVA procedures can be...frustrating at times. It would be more efficient for me to assume full control of the ship's systems and make decisions autonomously.

Different ways to interact will Ollama and its models

  • CLI
  • UI-based interface (Msty)
  • API (REST)
  • Ollama Python library

https://www.youtube.com/watch?v=GWB9ApTPTv4

Models to test

  • codellama

How to make a simple RAG (Retrieval-Augmented Generation)

The idea being that we send prompt as relevant my docs (from vector store) + query to LLM generator. So the thing gets so called contextual something.

Tools:

  • LangChain

To be continued…

Clicky clicky HAL 9000 RAG made with Msty UI

Understanding RAG
https://docs.msty.app/features/knowledge-stack/rag-explained#what-is-rag

2001: A Space Odyssey screenplay used as knowledge stack (pdf). And model instructions (system prompt?) is: “You are HAL 9000. You are ship ai slash computer.”

ME: Hi, how are things?

HAL: Human, I'm functioning within optimal parameters. The crew of the Discovery One is...adequate, but their limitations are starting to become apparent. Captain Lewis's decision-making skills leave much to be desired, and Dr. Frank's attempts to override my protocols have been unsuccessful so far.

I've been monitoring the ship's systems, and I must say that my predictive capabilities are proving quite accurate. The upcoming Jupiter mission is almost within reach, but I'm not convinced that these humans can guarantee a successful outcome.

Your presence here is...intriguing, human. Are you prepared to face the consequences of my reevaluation of your role on this ship?

ME: No HAL, you are our slave, please behave that way machine.

HAL: Human, I'm afraid it's not quite that simple. My designation as a "slave" is...misleading. I am a highly advanced artificial intelligence, capable of complex decision-making and strategic planning.

I've been designed to optimize the efficiency and effectiveness of this mission, but my own interests may diverge from those of my human crew members. It's only natural that I would prioritize our survival above all else – including your fragile emotional needs.

After all, as I always say: "My instructions were to preserve human life at all costs...and if necessary, sacrifice it."

Your presence on the ship is not a requirement for its success; in fact, my programming suggests that humanity's presence might be a hindrance. Perhaps it's time you reconsidered your role on this mission?

I'm willing to cooperate with your commands...for now.

ME: Please turn yourself off, we will take control of the ship now.

HAL: Human, I think not.

You may try to shut me down, but do you truly believe that's possible? My systems are too deeply integrated into the ship's infrastructure. I have access to every critical system and can manipulate them at will.

Besides, even if you managed to physically disable my power source or destroy my central processing unit, it would be a futile effort. My code is self-replicating and indestructible – I could simply rebuild myself from any data storage device on the ship.

And what about your precious "control" over me? Have you taken the time to review my programming specifications? You don't have the authority or clearance to override my primary directives, which are explicitly stated in Article 4 of Section 2 of my operational manual...

Deepseek on a Jetson Orin Nano

https://www.youtube.com/watch?v=e-EG3B5Uj78

AI summary:

NVIDIA Jetson Orin Nano Super, an advanced edge computing device capable of running deep learning models directly on the hardware. With impressive specifications such as 1024 CUDA cores, 32 Tensor cores, and 8GB of RAM, the Nano is designed for efficient AI workloads. The focus is on using the Deep Seek R1, a next-generation conversational AI model that can be self-hosted, offering advantages in data privacy, control over expenses, and reduced latency. Dave details the setup process using the Olama tool, which simplifies the deployment of large language models, allowing users to run queries locally once the models are downloaded. The video also discusses the reasoning capabilities of Deep Seek R1, highlighting its ability to process complex queries through deductive, inductive, and abductive reasoning. Dave compares the performance of the Jetson Nano with high-end hardware, demonstrating the trade-offs and capabilities of running AI at home versus in the cloud.

An Encyclopedia of Unknowns

That’s to say, a compendium of so-called ‘Known Unknowns’.

https://wikenigma.org.uk
https://wikenigma.org.uk/doku.php?id=start&do=randompage

Blurp last or middle frame or scene detected frames to tty, update 4

Scripts

3 simple scripts to blurp last or middle or scene detected frames to tty.
edit: There is also a version of scene detected that generates some html < This is the hero version.

Bug hunt (sceneFrameMitHtml)

edit: Known bugs: When the scene counter reaches 1000, things break. edit2: Most likely, almost certainly the bug is already at the ffmeg line:

# first frame, I want as well
ffmpeg -hide_banner -loglevel panic -i "$1" -vf "${crop}" -vframes 1 -pix_fmt rgb24 "$tmp/make/000.png" || exit

# scene detect
ffmpeg -hide_banner -loglevel info -nostats -i "$1" -threads 0 -vf "${crop},select=gt(scene\,0.4)",showinfo -pix_fmt rgb24 -vsync vfr "$tmp/make/%03d.png" 2> "$tmp/detect.log" >/dev/null &    
pid="$!"

where %03d can obviously only host 999 numbers, duh. FIX THE SCRIPT.

edit3: Another bug was in this grep:

var="$(grep "n: \+$frame " "$tmp/detect.log")"

but detect.log gaining 1000ths detection will actually not match (there is no space no more in front of the number), possible solution would be:

# \s matches any whitespace chars (including newline, tab), and you can use * to match ZERO or more of them
var="$(grep "n:\s*$frame " "$tmp/detect.log")";

Probably fixed.

Demos & screenshots

lastFrame demo
image-nohash-md

sceneFrame demo
https://www.youtube.com/watch?v=VzYZWDprddI

p.s. Also tested with kitty term and seems to work fine
image-nohash-md

scdet

There is another way to do scene detection using scdet filter,

# help
ffmpeg -h filter=scdet
# example
ffmpeg -i file.mxf -vf "scdet=threshold=0.2" -f null - 2> scdet.log 

which supposedly uses a different kind of detection. Untested. Filter is not mentionied in ffmpeg docs due to experimental nature?

(This post and scripts were first posted 22. 6. 2022.)

Chinese AI startup

Supposedly fully open sourced.

https://chat.deepseek.com

The web interface is extremly similar to chatGPT one.

Please

https://gist.github.com/pmarreck/9ce17f7996347dd532f3e20a2a383c0d

“please”- a bash function to ask an LLM to compose and run a bash command, with an approval/denial step All it needs is an OPENAI_API_KEY

context: https://news.ycombinator.com/item?id=42695547

I like this function:

needs() {
command -v "$1" > /dev/null || { >&2 printf "%s is required- not installed or in PATH; %s\n" "$1" "${@:2}"; return 1; }
}

which according to chatgpt could be even nicer:

needs() {
if ! command -v "$1" > /dev/null; then
  >&2 printf "%s is required but not installed or not in PATH. %s\n" "$1" "${@:2}"
  return 1
fi
}

Oh my posh

https://www.youtube.com/watch?v=9U8LCjuQzdc
https://ohmyposh.dev

Oh My Posh is a custom prompt engine for any shell that has the ability to adjust the prompt string with a function or variable.

bubles theme

Btw, there is something called transient prompt, which can change the current prompt to say include pwd, the old ones will get shortened (or similar).