Update README.md

README.md

@@ -11,3 +11,255 @@ base_model:
 see [webbigdata/FanFic-Illustrator](https://huggingface.co/webbigdata/FanFic-Illustrator).
 
 later, I will add sample script.
+
+
+```
+python launch.py --api --no-half-vae --medvram --xformers &
+```
+
+
+```
+./llama.cpp/build/bin/llama-server -m FanFic-Illustrator-Q6_K-f16.gguf -c 5000
+```
+
+
+
+sample client_script.py
+```
+import transformers
+from PIL import Image
+import io
+import base64
+import requests
+import json
+from transformers import AutoTokenizer
+tokenizer = AutoTokenizer.from_pretrained("webbigdata/FanFic-Illustrator")
+
+COMMON_QUALITY_PROMPT = ", masterpiece, high score, great score, absurdres"
+COMMON_NEGATIVE_PROMPT = "lowres, bad anatomy, bad hands, text, error, missing finger, extra digits, fewer digits, cropped, worst quality, low quality, low score, bad score, average score, signature, watermark, username, blurry"
+
+
+def generate_image(prompt, negative_prompt, output_path):
+    url = "http://localhost:7860/sdapi/v1/txt2img"
+
+    prompt = prompt.replace("(", "\\(").replace(")", "\\)")
+
+    payload = {
+        "prompt": prompt,
+        "negative_prompt": negative_prompt,
+        "steps": 28,
+        "cfg_scale": 5,
+        "width": 832,
+        "height": 1216,
+        "sampler_name": "Euler a",
+        "sampler_index": "Euler a",
+        "override_settings": {
+            "sd_model_checkpoint": "animagine-xl-4.0-opt.safetensors",
+            "sd_vae": "sdxl_vae.safetensors"
+        },
+        "enable_hr": False,
+        "batch_size": 1,
+        "do_not_save_samples": True,
+        "seed": -1,
+    }
+
+    response = requests.post(url=url, json=payload)
+    response.raise_for_status()
+    r = response.json()
+
+    image = Image.open(io.BytesIO(base64.b64decode(r['images'][0])))
+    image.save(output_path)
+
+    return True
+
+
+def get_novel_data():
+    return """Alice was beginning to get very tired of sitting by her sister on the bank, and of having nothing to do: once or twice she had peeped into the book her sister was reading, but it had no pictures or conversations in it, "and what is the use of a book," thought Alice "without pictures or conversations?"
+So she was considering in her own mind (as well as she could, for the hot day made her feel very sleepy and stupid), whether the pleasure of making a daisy-chain would be worth the trouble of getting up and picking the daisies, when suddenly a White Rabbit with pink eyes ran close by her.
+There was nothing so very remarkable in that; nor did Alice think it so very much out of the way to hear the Rabbit say to itself, "Oh dear! Oh dear! I shall be late!" (when she thought it over afterwards, it occurred to her that she ought to have wondered at this, but at the time it all seemed quite natural); but when the Rabbit actually took a watch out of its waistcoat-pocket, and looked at it, and then hurried on, Alice started to her feet, for it flashed across her mind that she had never before seen a rabbit with either a waistcoat-pocket, or a watch to take out of it, and burning with curiosity, she ran across the field after it, and fortunately was just in time to see it pop down a large rabbit-hole under the hedge.
+In another moment down went Alice after it, never once considering how in the world she was to get out again.
+The rabbit-hole went straight on like a tunnel for some way, and then dipped suddenly down, so suddenly that Alice had not a moment to think about stopping herself before she found herself falling down a very deep well."""
+
+if __name__ == "__main__":
+
+    novel_text = get_novel_data()
+    category = "general"
+    series_name = "alice in wonderland"
+    series_description = "A fantasy novel written by Lewis Carroll (Charles Lutwidge Dodgson) in 1865. The story follows a young girl named Alice who falls through a rabbit hole into a fantastical world populated by peculiar anthropomorphic creatures. The tale plays with logic, giving the story lasting popularity with adults as well as children. It is considered one of the best examples of the literary nonsense genre and its narrative, structure, characters, and imagery have been enormously influential in both popular culture and literature."
+    character_name = "alice (alice in wonderland)"
+    character_description = "A curious, polite, and logical seven-year-old girl with an imagination and a sense of wonder. Alice finds herself in a world that challenges her perceptions of reality and proper behavior. Throughout her journey in Wonderland, she encounters absurd situations and characters that test her patience and understanding. Despite her confusion, Alice maintains her rational thinking and often attempts to apply real-world logic to the nonsensical Wonderland. She is characterized by her curiosity, bravery, and her struggle to maintain her identity in a world where nothing makes sense."
+
+    system_prompt = "あなたは文章の一説を指定ジャンル・キャラクターが登場するシーンに書き換え、そのシーンに合った挿絵を作成するために画像生成AI用プロンプ���を作成する優秀なプロンプトエンジニアです"
+
+    user_prompt = f"""### 小説のコンテキストを補足する情報
+content category: {category}
+series name: {series_name}
+series description: {series_description}
+character name: {character_name}
+character description: {character_description}
+
+### 小説データ
+{novel_text}
+
+まず<think>内で以下のように思考を整理します。
+
+<think>
+concept: イラストのコンセプトを考えます。小説の内容から主題、設定、雰囲気を理解し、どのようなイラストが最も適切か、全体の構成を考えます
+- 人数: 挿絵の中に登場させる人数を考えます。作品に登場する人物の数や重要性を考慮し、メインで描くべき人物やサブキャラクターについても検討してください
+- キャラクター名/シリーズ名: 既存作品のキャラクター/シリーズか、オリジナル作品かを考えます。既存作品の場合は、原作の設定や特徴を尊重した表現方法も考慮してください
+- ポーズ/構図: ポーズ/構図指定に使うタグを考えます。物語の場面において、キャラクターがどのような体勢/状況にあるのか、どのアングルから描くと効果的かを検討してください
+- 背景/環境: 背景/環境指定に使うタグを考えます。物語の舞台設定や時間帯、天候など、雰囲気を表現するために必要な背景要素を詳しく考えてください
+- 描画スタイル/テクニック: 描画スタイル/テクニックに使うタグを考えます。物語のジャンルや雰囲気に合わせて、どのような画風や技法が適しているかを検討してください
+- 身体的特徴/画面上の物体: 身体的特徴/画面上の物体に関連するタグを考えます。キャラクターの外見的特徴や、シーンに必要な小道具、アイテムなどを詳細に考えてください
+</think>
+
+改行の場所も含めて、この順序と書式を厳密に守ってください。
+各項目は上記の順序と書式で記述してください。具体的かつ詳細に説明し、十分な長さで考察してください（<think>タグ全体で600-800文字程度が望ましいです）
+
+その後、思考結果に基づき<prompt>内に英単語を18単語ほどカンマで区切って出力してください。キャラクター名/シリーズ名は指定されていたら必ず含めます
+日本語は使用しないでください。 最も重要で適切なタグを選び、有効なプロンプトとなるよう考慮してください
+
+### 使用可能な英単語
+出力時には以下のタグを優先して使用し、足りない場合は一般的な英単語で補足します
+masterpiece, best quality, highresなどの品質に関連するタグは後工程で付与するのでつけてはいけません
+
+**人数/性別**:
+- 風景や動物を中心に描画する時: no_human
+- 女性の人数: 1girl, 2girls, 3girls, multiple girls
+- 男性の人数: 1boy, 2boys, 3boys, multiple boys
+- 1girlや1boy指定時にキャラクター中心の構図にするために追加で指定: solo
+
+**ポーズ/構図**:
+- 視点: from above, from behind, from below, looking at viewer, straight-on, looking at another, looking back, out of frame, on back, from side, looking to the side, feet out of frame, sideways, three quarter view, looking up, looking down, looking ahead, dutch angle, high up, from outside, pov, vanishing point
+- 姿勢/行動: battle, chasing, fighting, leaning, running, sitting, squatting, standing, walking, arm up, arms up, against wall, against tree, holding, spread legs, lying, straddling, flying, holding weapon, clothes lift, hand on own cheek, scar on cheek, hand on another's cheek, kissing cheek, cheek-to-cheek, bandaid on cheek, finger to cheek, hands on another's cheeks, hand on own hip, hand over face, v, kneeling, arabesque (pose), body roll, indian style, standing on one leg, hugging own legs, seiza, nuzzle, unsheathing, holding weapon, holding sword, holding gun, trembling
+
+**背景/環境**:
+- 構図/芸術ジャンル: landscape, portrait, still life, group shot, cowboy shot, upper body, full body, detailed face, depth of field, intricate details, cinematic lighting, detailed background, detailed, extremely detailed, perfect composition, detailed face, solo focus, detailed face and body, character focus, intricate, sharp focus, male focus
+- 色彩/装飾: greyscale, sepia, blue theme, flat color, high contrast, limited palette, border, cinematic, scenery, rendered, contrast, rich contrast, volumetric lighting, high contrast, glowing
+- 背景/風景: checkered background, simple background, indoors, outdoors, jungle, mountain, beach, forest, city, school, cafe, white background, sky
+- 時間帯: day, night, twilight, morning, sunset, dawn, dusk
+- 天気: sunny, rain, snow, cloud, storm, wind, fogg
+
+**描画スタイル/テクニック**:
+- 技法: 3D, oekaki, pixel art, sketch, watercolor, oil painting, digital art, illustration, photorealistic, anime, monochrome, retro color, source anime, cg, realistic
+- 表現手法: animalization, personification, science fiction, cyberpunk, steampunk, fantasy, dark novel style, anime style, realistic style, graphic novel style, comic, concept art
+- 媒体/伝統的技法: traditional media, marker (medium), watercolor (medium), graphite (medium), official art, sketch, artbook, cover
+- 絵柄の年代(指定された時のみ利用): newest, year 1980, year 2000, year 2010, year 1990, year 2020
+
+**身体的特徴/画面上の物体**:
+- キャラクター属性/職業/クラス: student, teacher, soldier, knight, wizard, ninja, doctor, artist, musician, athlete, virtual youtuber, chibi, maid, miko
+- 表情: angry, blush stickers, drunk, grin, aroused, happy, sad, smile, laugh, crying, surprised, worried, nervous, serious, drunk, blush, aroused, :d, tongue out, sweatdrop, tongue out, :o, tears, tearing up, scared
+
+- 身体的特徴: {{'髪型/髪色': ['long hair', 'short hair', 'twintails', 'ponytail', 'braid', 'bun', 'curly hair', 'straight hair', 'messy hair', 'blonde hair', 'black hair', 'brown hair', 'red hair', 'blue hair', 'green hair', 'white hair', 'purple hair', 'grey hair', 'ahoge', 'sidelocks', 'side ponytail', 'perfect hair', 'tail', 'multicolored hair', 'wavy hair', 'bangs', 'blunt bangs', 'twintails', 'hair between eyes', 'very long hair', 'braid', 'curly hair', 'braided ponytail', 'hand in own hair', 'hair over one eye', 'hair flower', 'two-tone hair', 'streaked hair', 'two side up'], '目の色': ['blue eyes', 'brown eyes', 'green eyes', 'red eyes', 'black eyes', 'purple eyes', 'yellow eyes', 'heterochromia', 'detailed eyes', 'glowing eyes', 'beatiful eyes', 'closed eyes', 'one eye closed'], '身体部位': ['bare shoulders', 'bare arms', 'bare legs', 'barefoot', 'abs', 'flat chest', 'small breasts', 'medium breasts', 'asymmetrical breasts', 'pointy breasts', 'sagging breasts', 'clenched teeth', 'pointy ears', 'perfect anatomy', 'closed mouth', 'long sleeves', 'open mouth', 'pale skin', 'collarbone', 'midriff', 'perfect anatomy', 'bare arms', 'thighs', 'parted lips', 'tongue', 'tanlines', 'dot nose', 'goggles on head', 'armpits', 'nail polish', 'mole', 'feet', 'lips', 'dark-skinned female', 'zettai ryouiki', 'shiny skin'], '身体部位(獣人、擬人化時のみ使用)': ['animal ears', 'cat ears', 'horse ears', 'horse girl', 'fang', 'teeth', 'horns', 'tail'], '服装/装飾品': ['uniform', 'suit', 'dress', 'casual wear', 'formal wear', 'belt', 'detached sleeves', 'swimsuit', 'kimono', 'armor', 'hat', 'glasses', 'white shirt', 'shirt', 'jewelry', 'necklace', 'earrings', 'bracelet', 'watch', 'ribbon', 'hair ribbon', 'scarf', 'gloves', 'boots', 'high heels', 'hair ornament', 'jacket', 'glasses', 'skirt', 'long sleeves', 'short sleeves', 'thighhighs', 'underwear', 'school uniform', 'swimsuit', 'panties', 'hair bow', 'bikini', 'miniskirt', 'fingerless gloves', 'bowtie', 'serafuku', 'japanese clothes', 'choker', 'pants', 'wings', 'open clothes', 'pantyhose', 'pleated skirt', 'frills', 'necktie', 'shorts', 'miko', 'collared shirt', 'leather armor', 'hairband', 'shoes', 'sleeveless', 'alternate costume', 'socks', 'fingering', 'denim shorts', 'epaulettes', 'santa costume', 'ribbon-trimmed sleeves', 'black bowtie', 'gym uniform', 'white bra', 'angel wings', 'crossdressing', 'cuffs', 'halo', 'high heels', 'apron', 'red bow', 'vest', 'open jacket', 'white panties', 'leotard', 'coat', 'black jacket', 'high heels', 'black pantyhose', 'see-through', 'miniskirt', 'elbow gloves', 'wide sleeves', 'white thighhighs', 'fur trim', 'plaid', 'one-piece swimsuit', 'maid headdress', 'ascot', 'high-waist skirt']}}
+- 体液: blood, saliva, sweat, tears
+- 前景/持ち物/操作物: sword, katana, sheath, gun, book, phone, bag, umbrella, instrument, vehicle, food, drink, guitar, piano, violin, drums, flute, car, bicycle, motorcycle, airplane, ship, flower, weapon, heart, speech bubble, carriage, locomotive, shovel
+- 生物: dog, cat, horse, bird, fish, crab, dragon, unicorn, monster, fox, wolf, bear, tiger, lion, dragon, fairy, ghost, zombie, vampire
+
+**性的表現（sensitive, nsfw, explicitのいずれかを指定した時のみ使用可）**:
+- 身体部位女性専用: cleavage, backboob, sideboob, underboob, navel, huge breasts, large breasts
+- 身体部位男性専用: topless male, necktie between pectorals, loose necktie, bare pectorals, male underwear, fundoshi
+- 身体部位共通: open shirt, unbuttoned shirt, seductive smile, bare back, groin, groin tendon, midriff
+
+### 出力
+"""
+    messages  = [
+        {"role": "system", "content": system_prompt},
+        {"role": "user", "content": user_prompt},
+    ]
+
+    prompt = tokenizer.apply_chat_template(
+            messages,
+            add_generation_prompt=True,
+            tokenize=False
+    )
+
+    payload = {
+            "prompt": prompt,
+            "n_predict": 280
+    }
+
+    url = "http://localhost:8080/completion"
+    headers = {
+        "Content-Type": "application/json"
+    }
+
+    response = requests.post(url, headers=headers, data=json.dumps(payload))
+    if response.status_code != 200:
+        print(f"Error: {response.text}")
+
+    model_response = response.json().get('content', '').strip()
+    print(f"model_response: {model_response}")
+
+    # モデルが生成したメッセージ部分だけを抽出
+    model_markers = ["assistant\n", "assistant:\n", "assitant\n"]
+
+    for marker in model_markers:
+        if marker in model_response:
+            model_response = model_response.split(marker)[-1].strip()
+            break
+
+    # 思考過程とプロンプトの抽出
+    thinking = ""
+    prompt_text = ""
+
+    if "<think>" in model_response and "</think>" in model_response:
+        thinking = model_response.split("<think>")[1].split("</think>")[0].strip()
+
+    def clean_prompt_text(text):
+        # 削除するタグのリスト
+        tags_to_remove = [
+            "masterpiece", "high score", "great score", "absurdres",
+            "highres", "original character", "original series",
+            "general", "sensitive", "nsfw", "explicit"
+        ]
+
+        # テキストを単語に分割して処理
+        words = []
+        current_words = text.split(',')
+
+        # 各単語をトリムして処理
+        for word in current_words:
+            word = word.strip()
+            # 空の単語はスキップ
+            if not word:
+                continue
+            # 削除対象のタグかチェック
+            if any(tag == word.lower() for tag in tags_to_remove):
+                continue
+            # まだ追加されていない単語のみ追加（重複排除）
+            if word not in words:
+                words.append(word)
+
+        # カンマで結合して返す
+        return ', '.join(words)
+
+    if "<prompt>" in model_response:
+        if "</prompt>" in model_response:
+            prompt_text = model_response.split("<prompt>")[1].split("</prompt>")[0].strip()
+        else:
+            prompt_text = model_response.split("<prompt>")[1].strip()
+
+        prompt_text = clean_prompt_text(prompt_text)
+    else:
+        prompt_text = f"1girl, {character_name}, {series_name}, anime style, highres"
+
+    prompt_text = prompt_text + f", {category}{COMMON_QUALITY_PROMPT}"
+
+    print(f"prompt for image: {prompt_text}")
+    generate_image(prompt_text, COMMON_NEGATIVE_PROMPT, "./test_image.png")
+
+```
+
+```
+model_response: <think>
+concept: アリスがワホットで落とられて、洞窟の中で呆然と立ち尽くしている場面。驚きと恐怖で表情を浮かべている。
+- 人数: 1girl
+- キャラクター名/シリーズ名: alice (alice in wonderland), alice in wonderland
+- ポーズ/構図: standing, surprised, looking at viewer
+- 背景/環境: indoors, cave, dark
+- 描画スタイル/テクニック: anime
+- 身体的特徴/画面上の物体: long hair, dress, worried, book
+</think>
+<prompt>1girl, alice (alice in wonderland), alice in wonderland, standing, surprised, looking at viewer, indoors, cave, dark, anime, long hair, dress, worried, book, general</prompt>
+```
+