66 "colab_type" : " text"
77 },
88 "source" : [
9- " # Segment Anything Model with \ud83e\udd17 Transformers \n "
9+ " # Segment Anything Model with 🤗Transformers \n "
1010 " \n "
1111 " **Authors:** [Merve Noyan](https://twitter.com/mervenoyann) & [Sayak Paul](https://twitter.com/RisingSayak)<br>\n "
1212 " **Date created:** 2023/07/11<br>\n "
1313 " **Last modified:** 2023/07/11<br>\n "
14- " **Description:** Fine-tuning Segment Anything Model using Keras and \ud83e\udd17 Transformers."
14+ " **Description:** Fine-tuning Segment Anything Model using Keras and 🤗 Transformers."
1515 ]
1616 },
1717 {
4545 " segmentation and edge detection. The goal of SAM is to enable all of these downstream\n "
4646 " segmentation tasks through prompting.\n "
4747 " \n "
48- " In this example, we'll learn how to use the SAM model from \ud83e\udd17 Transformers for performing\n "
49- " inference and fine-tuning.\n "
50- " "
48+ " In this example, we'll learn how to use the SAM model from 🤗 Transformers for performing\n "
49+ " inference and fine-tuning.\n "
5150 ]
5251 },
5352 {
6160 },
6261 {
6362 "cell_type" : " code"
64- "execution_count" : 0 ,
63+ "execution_count" : null ,
6564 "metadata" : {
6665 "colab_type" : " code"
6766 },
8180 },
8281 {
8382 "cell_type" : " code"
84- "execution_count" : 0 ,
83+ "execution_count" : null ,
8584 "metadata" : {
8685 "colab_type" : " code"
8786 },
109108 " \n "
110109 " SAM has the following components:\n "
111110 " \n "
112- " |\n "
113- " \n "
111+ " |  |\n "
114112 " |:--:|\n "
115- " | Image taken from the official\n "
116- " [SAM blog post](https://ai.facebook.com/blog/segment-anything-foundation-model-image-segmentation/) |\n "
117- " |"
113+ " | Image taken from the official [SAM blog post](https://ai.facebook.com/blog/segment-anything-foundation-model-image-segmentation/) |\n "
118114 ]
119115 },
120116 {
174170 },
175171 {
176172 "cell_type" : " code"
177- "execution_count" : 0 ,
173+ "execution_count" : null ,
178174 "metadata" : {
179175 "colab_type" : " code"
180176 },
197193 },
198194 {
199195 "cell_type" : " code"
200- "execution_count" : 0 ,
196+ "execution_count" : null ,
201197 "metadata" : {
202198 "colab_type" : " code"
203199 },
315311 " show_mask(mask, axes[i])\n "
316312 " axes[i].title.set_text(f\" Mask {i+1}, Score: {score.numpy().item():.3f}\" )\n "
317313 " axes[i].axis(\" off\" )\n "
318- " plt.show()\n "
319- " "
314+ " plt.show()\n "
320315 ]
321316 },
322317 {
333328 },
334329 {
335330 "cell_type" : " code"
336- "execution_count" : 0 ,
331+ "execution_count" : null ,
337332 "metadata" : {
338333 "colab_type" : " code"
339334 },
357352 },
358353 {
359354 "cell_type" : " code"
360- "execution_count" : 0 ,
355+ "execution_count" : null ,
361356 "metadata" : {
362357 "colab_type" : " code"
363358 },
380375 },
381376 {
382377 "cell_type" : " code"
383- "execution_count" : 0 ,
378+ "execution_count" : null ,
384379 "metadata" : {
385380 "colab_type" : " code"
386381 },
416411 },
417412 {
418413 "cell_type" : " code"
419- "execution_count" : 0 ,
414+ "execution_count" : null ,
420415 "metadata" : {
421416 "colab_type" : " code"
422417 },
443438 " As can be noticed, all the masks are _valid_ masks for the point prompt we provided.\n "
444439 " \n "
445440 " SAM is flexible enough to support different visual prompts and we encourage you to check\n "
446- " out [this\n "
447- " notebook](https://github.com/huggingface/notebooks/blob/main/examples/segment_anything.ipy\n "
448- " nb) to know more about them!"
441+ " out [this notebook](https://github.com/huggingface/notebooks/blob/main/examples/segment_anything.ipynb) to know more about them!"
449442 ]
450443 },
451444 {
467460 },
468461 {
469462 "cell_type" : " code"
470- "execution_count" : 0 ,
463+ "execution_count" : null ,
471464 "metadata" : {
472465 "colab_type" : " code"
473466 },
494487 },
495488 {
496489 "cell_type" : " code"
497- "execution_count" : 0 ,
490+ "execution_count" : null ,
498491 "metadata" : {
499492 "colab_type" : " code"
500493 },
553546 },
554547 {
555548 "cell_type" : " code"
556- "execution_count" : 0 ,
549+ "execution_count" : null ,
557550 "metadata" : {
558551 "colab_type" : " code"
559552 },
606599 " y_max = min(H, y_max + np.random.randint(0, 20))\n "
607600 " bbox = [x_min, y_min, x_max, y_max]\n "
608601 " \n "
609- " return bbox\n "
610- " "
602+ " return bbox\n "
611603 ]
612604 },
613605 {
634626 },
635627 {
636628 "cell_type" : " code"
637- "execution_count" : 0 ,
629+ "execution_count" : null ,
638630 "metadata" : {
639631 "colab_type" : " code"
640632 },
667659 },
668660 {
669661 "cell_type" : " code"
670- "execution_count" : 0 ,
662+ "execution_count" : null ,
671663 "metadata" : {
672664 "colab_type" : " code"
673665 },
696688 },
697689 {
698690 "cell_type" : " code"
699- "execution_count" : 0 ,
691+ "execution_count" : null ,
700692 "metadata" : {
701693 "colab_type" : " code"
702694 },
723715 },
724716 "source" : [
725717 " We will now write DICE loss. This implementation is based on\n "
726- " [MONAI DICE loss](https://docs.monai.io/en/stable/_modules/monai/ losses/dice .html#DiceLoss )."
718+ " [MONAI DICE loss](https://docs.monai.io/en/stable/losses.html#diceloss )."
727719 ]
728720 },
729721 {
730722 "cell_type" : " code"
731- "execution_count" : 0 ,
723+ "execution_count" : null ,
732724 "metadata" : {
733725 "colab_type" : " code"
734726 },
752744 " loss = 1.0 - (2.0 * intersection + 1e-5) / (denominator + 1e-5)\n "
753745 " loss = tf.reduce_mean(loss)\n "
754746 " \n "
755- " return loss\n "
756- " "
747+ " return loss\n "
757748 ]
758749 },
759750 {
762753 "colab_type" : " text"
763754 },
764755 "source" : [
765- " ##\u00a0 Fine -tuning SAM\n "
756+ " ## Fine -tuning SAM\n "
766757 " \n "
767758 " We will now fine-tune SAM's decoder part. We will freeze the vision encoder and prompt\n "
768759 " encoder layers."
769760 ]
770761 },
771762 {
772763 "cell_type" : " code"
773- "execution_count" : 0 ,
764+ "execution_count" : null ,
774765 "metadata" : {
775766 "colab_type" : " code"
776767 },
806797 " grads = tape.gradient(loss, trainable_vars)\n "
807798 " optimizer.apply_gradients(zip(grads, trainable_vars))\n "
808799 " \n "
809- " return loss\n "
810- " "
800+ " return loss\n "
811801 ]
812802 },
813803 {
822812 },
823813 {
824814 "cell_type" : " code"
825- "execution_count" : 0 ,
815+ "execution_count" : null ,
826816 "metadata" : {
827817 "colab_type" : " code"
828818 },
859849 },
860850 {
861851 "cell_type" : " code"
862- "execution_count" : 0 ,
852+ "execution_count" : null ,
863853 "metadata" : {
864854 "colab_type" : " code"
865855 },
880870 },
881871 {
882872 "cell_type" : " code"
883- "execution_count" : 0 ,
873+ "execution_count" : null ,
884874 "metadata" : {
885875 "colab_type" : " code"
886876 },
906896 },
907897 {
908898 "cell_type" : " code"
909- "execution_count" : 0 ,
899+ "execution_count" : null ,
910900 "metadata" : {
911901 "colab_type" : " code"
912902 },
947937 },
948938 "nbformat" : 4 ,
949939 "nbformat_minor" : 0
950- }
940+ }
0 commit comments