【Java】生产者消费者模式的实现

article/2025/9/29 9:27:02

前言

生产者消费者问题是线程模型中的经典问题：生产者和消费者在同一时间段内共用同一存储空间，生产者向空间里生产数据，而消费者取走数据。

阻塞队列就相当于一个缓冲区，平衡了生产者和消费者的处理能力。这个阻塞队列就是用来给生产者和消费者解耦的。

wait/notify方法

首先，我们搞清楚Thread.sleep()方法和Object.wait()、Object.notify()方法的区别。根据这篇文章java sleep和wait的区别的疑惑?

  1. sleep()是Thread类的方法；而wait()，notify()，notifyAll()是Object类中定义的方法；尽管这两个方法都会影响线程的执行行为，但是本质上是有区别的。

  2. Thread.sleep()不会导致锁行为的改变，如果当前线程是拥有锁的，那么Thread.sleep()不会让线程释放锁。如果能够帮助你记忆的话，可以简单认为和锁相关的方法都定义在Object类中，因此调用Thread.sleep()是不会影响锁的相关行为。

  3. Thread.sleep和Object.wait都会暂停当前的线程，对于CPU资源来说，不管是哪种方式暂停的线程，都表示它暂时不再需要CPU的执行时间。OS会将执行时间分配给其它线程。区别是调用wait后，需要别的线程执行notify/notifyAll才能够重新获得CPU执行时间。

线程状态图：

  • Thread.sleep()让线程从 【running】 -> 【阻塞态】 时间结束/interrupt -> 【runnable】
  • Object.wait()让线程从 【running】 -> 【等待队列】notify -> 【锁池】 -> 【runnable】

实现生产者消费者模型

生产者消费者问题是研究多线程程序时绕不开的经典问题之一，它描述是有一块缓冲区作为仓库，生产者可以将产品放入仓库，消费者则可以从仓库中取走产品。在Java中一共有四种方法支持同步，其中前三个是同步方法，一个是管道方法。

（1）Object的wait() / notify()方法
（2）Lock和Condition的await() / signal()方法
（3）BlockingQueue阻塞队列方法
（4）PipedInputStream / PipedOutputStream

本文只介绍最常用的前三种，第四种暂不做讨论。源代码在这里：Java实现生产者消费者模型

1. 使用Object的wait() / notify()方法

wait()/ nofity()方法是基类Object的两个方法，也就意味着所有Java类都会拥有这两个方法，这样，我们就可以为任何对象实现同步机制。

  • wait()：当缓冲区已满/空时，生产者/消费者线程停止自己的执行，放弃锁，使自己处于等待状态，让其他线程执行。
  • notify()：当生产者/消费者向缓冲区放入/取出一个产品时，向其他等待的线程发出可执行的通知，同时放弃锁，使自己处于等待状态。
/*** 生产者消费者模式：使用Object.wait() / notify()方法实现*/
public class ProducerConsumer {private static final int CAPACITY = 5;public static void main(String args[]){Queue<Integer> queue = new LinkedList<Integer>();Thread producer1 = new Producer("P-1", queue, CAPACITY);Thread producer2 = new Producer("P-2", queue, CAPACITY);Thread consumer1 = new Consumer("C1", queue, CAPACITY);Thread consumer2 = new Consumer("C2", queue, CAPACITY);Thread consumer3 = new Consumer("C3", queue, CAPACITY);producer1.start();producer2.start();consumer1.start();consumer2.start();consumer3.start();}/*** 生产者*/public static class Producer extends Thread{private Queue<Integer> queue;String name;int maxSize;int i = 0;public Producer(String name, Queue<Integer> queue, int maxSize){super(name);this.name = name;this.queue = queue;this.maxSize = maxSize;}@Overridepublic void run(){while(true){synchronized(queue){while(queue.size() == maxSize){try {System.out .println("Queue is full, Producer[" + name + "] thread waiting for " + "consumer to take something from queue.");queue.wait();} catch (Exception ex) {ex.printStackTrace();}}System.out.println("[" + name + "] Producing value : +" + i);queue.offer(i++);queue.notifyAll();try {Thread.sleep(new Random().nextInt(1000));} catch (InterruptedException e) {e.printStackTrace();}}}}}/*** 消费者*/public static class Consumer extends Thread{private Queue<Integer> queue;String name;int maxSize;public Consumer(String name, Queue<Integer> queue, int maxSize){super(name);this.name = name;this.queue = queue;this.maxSize = maxSize;}@Overridepublic void run(){while(true){synchronized(queue){while(queue.isEmpty()){try {System.out.println("Queue is empty, Consumer[" + name + "] thread is waiting for Producer");queue.wait();} catch (Exception ex) {ex.printStackTrace();}}int x = queue.poll();System.out.println("[" + name + "] Consuming value : " + x);queue.notifyAll();try {Thread.sleep(new Random().nextInt(1000));} catch (InterruptedException e) {e.printStackTrace();}}}}}
}
注意要点

判断Queue大小为0或者大于等于queueSize时须使用 while (condition) {}，不能使用 if(condition) {}。其中 while(condition)循环，它又被叫做“自旋锁”。自旋锁以及wait()和notify()方法在线程通信这篇文章中有更加详细的介绍。为防止该线程没有收到notify()调用也从wait()中返回（也称作虚假唤醒），这个线程会重新去检查condition条件以决定当前是否可以安全地继续执行还是需要重新保持等待，而不是认为线程被唤醒了就可以安全地继续执行了。

输出日志如下：

[P-1] Producing value : +0
[P-1] Producing value : +1
[P-1] Producing value : +2
[P-1] Producing value : +3
[P-1] Producing value : +4
Queue is full, Producer[P-1] thread waiting for consumer to take something from queue.
[C3] Consuming value : 0
[C3] Consuming value : 1
[C3] Consuming value : 2
[C3] Consuming value : 3
[C3] Consuming value : 4
Queue is empty, Consumer[C3] thread is waiting for Producer
Queue is empty, Consumer[C2] thread is waiting for Producer
Queue is empty, Consumer[C1] thread is waiting for Producer
[P-2] Producing value : +0
[C1] Consuming value : 0
Queue is empty, Consumer[C1] thread is waiting for Producer
Queue is empty, Consumer[C2] thread is waiting for Producer
Queue is empty, Consumer[C3] thread is waiting for Producer
[P-1] Producing value : +5
[P-1] Producing value : +6
[P-1] Producing value : +7
[P-1] Producing value : +8
[P-1] Producing value : +9
Queue is full, Producer[P-1] thread waiting for consumer to take something from queue.
[C3] Consuming value : 5
[C3] Consuming value : 6
[C3] Consuming value : 7
[C3] Consuming value : 8
[C3] Consuming value : 9
Queue is empty, Consumer[C3] thread is waiting for Producer
Queue is empty, Consumer[C2] thread is waiting for Producer
Queue is empty, Consumer[C1] thread is waiting for Producer
[P-2] Producing value : +1
[C1] Consuming value : 1
Queue is empty, Consumer[C1] thread is waiting for Producer
Queue is empty, Consumer[C2] thread is waiting for Producer
Queue is empty, Consumer[C3] thread is waiting for Producer
[P-1] Producing value : +10
[P-1] Producing value : +11
[P-1] Producing value : +12
[P-1] Producing value : +13
[P-1] Producing value : +14
Queue is full, Producer[P-1] thread waiting for consumer to take something from queue.
[C3] Consuming value : 10
[C3] Consuming value : 11
[C3] Consuming value : 12
[C3] Consuming value : 13
[C3] Consuming value : 14
Queue is empty, Consumer[C3] thread is waiting for Producer
Queue is empty, Consumer[C2] thread is waiting for Producer
Queue is empty, Consumer[C1] thread is waiting for Producer
[P-2] Producing value : +2
[P-2] Producing value : +3
[P-2] Producing value : +4
[P-2] Producing value : +5
[P-2] Producing value : +6
Queue is full, Producer[P-2] thread waiting for consumer to take something from queue.
[C1] Consuming value : 2
[C1] Consuming value : 3
[C1] Consuming value : 4
[C1] Consuming value : 5
[C1] Consuming value : 6
Queue is empty, Consumer[C1] thread is waiting for Producer
Queue is empty, Consumer[C2] thread is waiting for Producer
Queue is empty, Consumer[C3] thread is waiting for Producer
[P-1] Producing value : +15
[C3] Consuming value : 15
Queue is empty, Consumer[C3] thread is waiting for Producer
Queue is empty, Consumer[C2] thread is waiting for Producer
Queue is empty, Consumer[C1] thread is waiting for Producer
[P-2] Producing value : +7
[P-2] Producing value : +8
[P-2] Producing value : +9

2. 使用Lock和Condition的await() / signal()方法

在JDK5.0之后，Java提供了更加健壮的线程处理机制，包括同步、锁定、线程池等，它们可以实现更细粒度的线程控制。Condition接口的await()和signal()就是其中用来做同步的两种方法，它们的功能基本上和Object的wait()/ nofity()相同，完全可以取代它们，但是它们和新引入的锁定机制Lock直接挂钩，具有更大的灵活性。通过在Lock对象上调用newCondition()方法，将条件变量和一个锁对象进行绑定，进而控制并发程序访问竞争资源的安全。下面来看代码：

/*** 生产者消费者模式：使用Lock和Condition实现* {@link java.util.concurrent.locks.Lock}* {@link java.util.concurrent.locks.Condition}*/
public class ProducerConsumerByLock {private static final int CAPACITY = 5;private static final Lock lock = new ReentrantLock();private static final Condition fullCondition = lock.newCondition();     //队列满的条件private static final Condition emptyCondition = lock.newCondition();        //队列空的条件public static void main(String args[]){Queue<Integer> queue = new LinkedList<Integer>();Thread producer1 = new Producer("P-1", queue, CAPACITY);Thread producer2 = new Producer("P-2", queue, CAPACITY);Thread consumer1 = new Consumer("C1", queue, CAPACITY);Thread consumer2 = new Consumer("C2", queue, CAPACITY);Thread consumer3 = new Consumer("C3", queue, CAPACITY);producer1.start();producer2.start();consumer1.start();consumer2.start();consumer3.start();}/*** 生产者*/public static class Producer extends Thread{private Queue<Integer> queue;String name;int maxSize;int i = 0;public Producer(String name, Queue<Integer> queue, int maxSize){super(name);this.name = name;this.queue = queue;this.maxSize = maxSize;}@Overridepublic void run(){while(true){//获得锁lock.lock();while(queue.size() == maxSize){try {System.out .println("Queue is full, Producer[" + name + "] thread waiting for " + "consumer to take something from queue.");//条件不满足，生产阻塞fullCondition.await();} catch (InterruptedException ex) {ex.printStackTrace();}}System.out.println("[" + name + "] Producing value : +" + i);queue.offer(i++);//唤醒其他所有生产者、消费者fullCondition.signalAll();emptyCondition.signalAll();//释放锁lock.unlock();try {Thread.sleep(new Random().nextInt(1000));} catch (InterruptedException e) {e.printStackTrace();}}}}/*** 消费者*/public static class Consumer extends Thread{private Queue<Integer> queue;String name;int maxSize;public Consumer(String name, Queue<Integer> queue, int maxSize){super(name);this.name = name;this.queue = queue;this.maxSize = maxSize;}@Overridepublic void run(){while(true){//获得锁lock.lock();while(queue.isEmpty()){try {System.out.println("Queue is empty, Consumer[" + name + "] thread is waiting for Producer");//条件不满足，消费阻塞emptyCondition.await();} catch (Exception ex) {ex.printStackTrace();}}int x = queue.poll();System.out.println("[" + name + "] Consuming value : " + x);//唤醒其他所有生产者、消费者fullCondition.signalAll();emptyCondition.signalAll();//释放锁lock.unlock();try {Thread.sleep(new Random().nextInt(1000));} catch (InterruptedException e) {e.printStackTrace();}}}}
}

输入日志如下：

[P-1] Producing value : +0
[C1] Consuming value : 0
Queue is empty, Consumer[C3] thread is waiting for Producer
Queue is empty, Consumer[C2] thread is waiting for Producer
[P-2] Producing value : +0
[C3] Consuming value : 0
Queue is empty, Consumer[C2] thread is waiting for Producer
Queue is empty, Consumer[C1] thread is waiting for Producer
[P-2] Producing value : +1
[C2] Consuming value : 1
Queue is empty, Consumer[C1] thread is waiting for Producer
Queue is empty, Consumer[C3] thread is waiting for Producer
[P-1] Producing value : +1
[C1] Consuming value : 1
Queue is empty, Consumer[C3] thread is waiting for Producer
[P-1] Producing value : +2
[C3] Consuming value : 2
Queue is empty, Consumer[C2] thread is waiting for Producer
[P-2] Producing value : +2
[C2] Consuming value : 2
Queue is empty, Consumer[C1] thread is waiting for Producer
Queue is empty, Consumer[C2] thread is waiting for Producer
[P-1] Producing value : +3
[C1] Consuming value : 3
Queue is empty, Consumer[C2] thread is waiting for Producer
Queue is empty, Consumer[C1] thread is waiting for Producer
Queue is empty, Consumer[C3] thread is waiting for Producer
[P-2] Producing value : +3
[C2] Consuming value : 3
Queue is empty, Consumer[C1] thread is waiting for Producer
Queue is empty, Consumer[C3] thread is waiting for Producer
Queue is empty, Consumer[C2] thread is waiting for Producer
[P-1] Producing value : +4
[C1] Consuming value : 4
Queue is empty, Consumer[C3] thread is waiting for Producer
Queue is empty, Consumer[C2] thread is waiting for Producer
Queue is empty, Consumer[C1] thread is waiting for Producer
[P-2] Producing value : +4
[C3] Consuming value : 4
Queue is empty, Consumer[C2] thread is waiting for Producer
Queue is empty, Consumer[C1] thread is waiting for Producer
[P-2] Producing value : +5
[C2] Consuming value : 5
Queue is empty, Consumer[C1] thread is waiting for Producer
Queue is empty, Consumer[C2] thread is waiting for Producer
[P-1] Producing value : +5
[C1] Consuming value : 5
Queue is empty, Consumer[C2] thread is waiting for Producer
Queue is empty, Consumer[C3] thread is waiting for Producer
[P-2] Producing value : +6
[C2] Consuming value : 6
Queue is empty, Consumer[C3] thread is waiting for Producer
[P-1] Producing value : +6
[C3] Consuming value : 6
Queue is empty, Consumer[C3] thread is waiting for Producer
Queue is empty, Consumer[C1] thread is waiting for Producer
[P-2] Producing value : +7
[C3] Consuming value : 7
Queue is empty, Consumer[C1] thread is waiting for Producer
[P-1] Producing value : +7
[C1] Consuming value : 7
Queue is empty, Consumer[C2] thread is waiting for Producer
[P-2] Producing value : +8
[C2] Consuming value : 8
[P-1] Producing value : +8
[C1] Consuming value : 8
[P-2] Producing value : +9
[C3] Consuming value : 9
[P-2] Producing value : +10
[C2] Consuming value : 10
[P-1] Producing value : +9
[P-1] Producing value : +10
[C1] Consuming value : 9
[P-2] Producing value : +11
[C3] Consuming value : 10
[C2] Consuming value : 11
[P-2] Producing value : +12
[C1] Consuming value : 12
[P-1] Producing value : +11
[C3] Consuming value : 11
[P-2] Producing value : +13
[C2] Consuming value : 13
Queue is empty, Consumer[C2] thread is waiting for Producer
Queue is empty, Consumer[C3] thread is waiting for Producer
[P-1] Producing value : +12
[C2] Consuming value : 12
Queue is empty, Consumer[C3] thread is waiting for Producer
[P-1] Producing value : +13
[C3] Consuming value : 13
Queue is empty, Consumer[C1] thread is waiting for Producer
Queue is empty, Consumer[C3] thread is waiting for Producer
[P-2] Producing value : +14
[C1] Consuming value : 14
Queue is empty, Consumer[C3] thread is waiting for Producer
Queue is empty, Consumer[C1] thread is waiting for Producer
[P-1] Producing value : +14
[C3] Consuming value : 14
Queue is empty, Consumer[C1] thread is waiting for Producer
[P-1] Producing value : +15
[C1] Consuming value : 15
[P-2] Producing value : +15
[P-1] Producing value : +16
[C3] Consuming value : 15
[P-2] Producing value : +16

3. 使用BlockingQueue阻塞队列方法

JDK 1.5 以后新增的 java.util.concurrent包新增了 BlockingQueue 接口。并提供了如下几种阻塞队列实现：

  • java.util.concurrent.ArrayBlockingQueue
  • java.util.concurrent.LinkedBlockingQueue
  • java.util.concurrent.SynchronousQueue
  • java.util.concurrent.PriorityBlockingQueue

实现生产者-消费者模型使用 ArrayBlockingQueue或者 LinkedBlockingQueue即可。

我们这里使用LinkedBlockingQueue，它是一个已经在内部实现了同步的队列，实现方式采用的是我们第2种await()/ signal()方法。它可以在生成对象时指定容量大小。它用于阻塞操作的是put()和take()方法。

  • put()方法：类似于我们上面的生产者线程，容量达到最大时，自动阻塞。
  • take()方法：类似于我们上面的消费者线程，容量为0时，自动阻塞。

我们可以跟进源码看一下LinkedBlockingQueue类的put()方法实现：

/** Main lock guarding all access */
final ReentrantLock lock = new ReentrantLock();/** Condition for waiting takes */
private final Condition notEmpty = lock.newCondition();/** Condition for waiting puts */
private final Condition notFull = lock.newCondition();public void put(E e) throws InterruptedException {putLast(e);
}public void putLast(E e) throws InterruptedException {if (e == null) throw new NullPointerException();Node<E> node = new Node<E>(e);final ReentrantLock lock = this.lock;lock.lock();try {while (!linkLast(node))notFull.await();} finally {lock.unlock();}
}

看到这里证实了它的实现方式采用的是我们第2种await()/ signal()方法。下面我们就使用它实现吧。

/*** 生产者消费者模式：使用{@link java.util.concurrent.BlockingQueue}实现*/
public class ProducerConsumerByBQ{private static final int CAPACITY = 5;public static void main(String args[]){LinkedBlockingDeque<Integer> blockingQueue = new LinkedBlockingDeque<Integer>(CAPACITY);Thread producer1 = new Producer("P-1", blockingQueue, CAPACITY);Thread producer2 = new Producer("P-2", blockingQueue, CAPACITY);Thread consumer1 = new Consumer("C1", blockingQueue, CAPACITY);Thread consumer2 = new Consumer("C2", blockingQueue, CAPACITY);Thread consumer3 = new Consumer("C3", blockingQueue, CAPACITY);producer1.start();producer2.start();consumer1.start();consumer2.start();consumer3.start();}/*** 生产者*/public static class Producer extends Thread{private LinkedBlockingDeque<Integer> blockingQueue;String name;int maxSize;int i = 0;public Producer(String name, LinkedBlockingDeque<Integer> queue, int maxSize){super(name);this.name = name;this.blockingQueue = queue;this.maxSize = maxSize;}@Overridepublic void run(){while(true){try {blockingQueue.put(i);System.out.println("[" + name + "] Producing value : +" + i);i++;//暂停最多1秒Thread.sleep(new Random().nextInt(1000));} catch (InterruptedException e) {e.printStackTrace();}}}}/*** 消费者*/public static class Consumer extends Thread{private LinkedBlockingDeque<Integer> blockingQueue;String name;int maxSize;public Consumer(String name, LinkedBlockingDeque<Integer> queue, int maxSize){super(name);this.name = name;this.blockingQueue = queue;this.maxSize = maxSize;}@Overridepublic void run(){while(true){try {int x = blockingQueue.take();System.out.println("[" + name + "] Consuming : " + x);//暂停最多1秒Thread.sleep(new Random().nextInt(1000));} catch (InterruptedException e) {e.printStackTrace();}}}}
}

输出日志如下：

[P-2] Producing value : +0
[P-1] Producing value : +0
[C1] Consuming : 0
[C3] Consuming : 0
[P-2] Producing value : +1
[C2] Consuming : 1
[P-2] Producing value : +2
[C1] Consuming : 2
[P-1] Producing value : +1
[C2] Consuming : 1
[P-1] Producing value : +2
[C3] Consuming : 2
[P-1] Producing value : +3
[C2] Consuming : 3
[P-2] Producing value : +3
[C1] Consuming : 3
[P-1] Producing value : +4
[C2] Consuming : 4
[P-2] Producing value : +4
[C3] Consuming : 4
[P-2] Producing value : +5
[C1] Consuming : 5
[P-1] Producing value : +5
[C2] Consuming : 5
[P-1] Producing value : +6
[C1] Consuming : 6
[P-2] Producing value : +6
[C2] Consuming : 6
[P-2] Producing value : +7
[C2] Consuming : 7
[P-1] Producing value : +7
[C1] Consuming : 7
[P-2] Producing value : +8
[C3] Consuming : 8
[P-2] Producing value : +9
[C2] Consuming : 9
[P-1] Producing value : +8
[C2] Consuming : 8
[P-2] Producing value : +10
[C1] Consuming : 10
[P-1] Producing value : +9
[C3] Consuming : 9
[P-1] Producing value : +10
[C2] Consuming : 10
[P-2] Producing value : +11
[C1] Consuming : 11
[C3] Consuming : 12
[P-2] Producing value : +12
[P-2] Producing value : +13
[C2] Consuming : 13
[P-1] Producing value : +11
[C3] Consuming : 11
[P-1] Producing value : +12
[C3] Consuming : 12
[P-2] Producing value : +14
[C1] Consuming : 14
[P-1] Producing value : +13
[C2] Consuming : 13
[P-2] Producing value : +15
[C3] Consuming : 15
[P-2] Producing value : +16
[C1] Consuming : 16
[P-1] Producing value : +14
[C3] Consuming : 14
[P-2] Producing value : +17
[C2] Consuming : 17

参考资料

  • Producer-Consumer solution using threads in Java
  • 生产者消费者问题 - 维基百科
  • 生产者/消费者问题的多种Java实现方式
  • 如何在 Java 中正确使用 wait, notify 和 notifyAll – 以生产者消费者模型为例
  • JAVA多线程之wait/notify
  • java sleep和wait的区别的疑惑?

http://chatgpt.dhexx.cn/article/gS42Fo48.shtml

相关文章

生产者/消费者模式

生产者/消费者模式

[0]&#xff1a;概述 今天打算来介绍一下“生产者&#xff0f;消费者模式”&#xff0c;这玩意儿在很多开发领域都能派上用场。由于该模式很重要&#xff0c;打算分几个帖子来介绍。今天这个帖子先来扫盲一把。如果你对这个模式已经比较了解&#xff0c;请跳过本扫盲帖&#x…
阅读更多...
（四）生产者消费者模式

（四）生产者消费者模式

&#xff08;一)生产者消费者模式原理&#xff1a; 在一个系统中&#xff0c;存在生产者和消费者两种角色&#xff0c;他们通过内存缓冲区进行通信&#xff0c;生产者生产消费者需要的资料&#xff0c;消费者把资料做成产品。生产消费者模式如下图&#xff1a; &#xff08;二…
阅读更多...
【C++】【设计模式之】生产者-消费者模型(理论讲解及实现)

【C++】【设计模式之】生产者-消费者模型(理论讲解及实现)

一、什么是生产者-消费者模型 1、简单理解生产者-消费者模型 假设有两个进程&#xff08;或线程&#xff09;A、B和一个固定大小的缓冲区&#xff0c;A进程生产数据放入缓冲区&#xff0c;B进程从缓冲区中取出数据进行计算&#xff0c;这就是一个简单的生产者-消费者模型。这里…
阅读更多...
设计模式——生产者消费者模式

设计模式——生产者消费者模式

1 基本概括 2 主要介绍 2.1 概念 生产者消费者模式是通过一个容器来解决生产者和消费者的强耦合问题。生产者和消费者彼此之间不直接通讯&#xff0c;而通过阻塞队列来进行通讯&#xff0c;所以生产者生产完数据之后不用等待消费者处理&#xff0c;直接扔给阻塞队列&#xff…
阅读更多...
生产者消费者模式三种实现方式

生产者消费者模式三种实现方式

目录 1.什么是生产者消费者模式&#xff1a;2.生产者消费者模型的实现&#xff1a;第一种&#xff1a;使用 synchronized和wait、notify第二种&#xff1a;使用 Lock和await、signal第三种&#xff1a;使用 阻塞队列 BlockingQueue 1.什么是生产者消费者模式&#xff1a; 生产…
阅读更多...
t-SNE算法

t-SNE算法

t-SNE(t-distributed stochastic neighbor embedding)是用于降维的一种机器学习算法&#xff0c;是由 Laurens van der Maaten 和 Geoffrey Hinton在 08 年提出来。t-SNE 是一种非线性降维算法&#xff0c;非常适用于高维数据降维到 2 维或者 3 维&#xff0c;进行可视化。在实…
阅读更多...
t-SNE概述

t-SNE概述

为了循序渐进, 先来学习SNE. SNE 无论是多维数据还是词向量, 都是一个个散落在空间中的点, 点与点之间距离近的, 就可以看作属于同一分类或近义词. 衡量两点距离有很多种手段, 但最常用的还是欧式距离, 所以欧氏距离与相似度的关系可以用某种公式近似表达, 这样就可以把空间信…
阅读更多...
机器学习笔记 - 什么是t-SNE？

机器学习笔记 - 什么是t-SNE？

1、t-SNE概述 t-Distributed Stochastic Neighbor Embedding (t-SNE) 是一种无监督的非线性技术，主要用于数据探索和高维数据的可视化。简单来说，t-SNE 让您对数据在高维空间中的排列方式有一种感觉或直觉。它由 Laurens van der Maatens 和 Geoffrey Hinton 于 2008 年提出。…
阅读更多...
可视化降维方法 t-SNE

可视化降维方法 t-SNE

本篇主要介绍很好的降维方法t-SNE的原理 详细介绍了困惑度perplexity对有效点的影响首先介绍了SNE然后在SNE的基础上进行改进&#xff1a;1.使用对称式。2.低维空间概率计算使用t分布 t-SNE&#xff08;t分布和SNE的组合&#xff09; 以前的方法有个问题&#xff1a;只考虑相…
阅读更多...
t-SNE非线性降维

t-SNE非线性降维

TSNE&#xff08;t-Distributed Stochastic Neighbor Embedding &#xff09;是对SNE的改进&#xff0c;SNE最早出现在2002年&#xff0c;改变了MDN和ISOMAP中基于距离不变的思想&#xff0c;将高维映射到低维的同时&#xff0c;尽量保证相互之间的分布概率不变&#xff0c;SNE…
阅读更多...
t-SNE原理及代码

t-SNE原理及代码

SNE 基本原理 SNE是通过仿射变换将数据点映射到概率分布上&#xff0c;主要包括两个步骤&#xff1a; 　&#xff11;) SNE构建一个高维对象之间的概率分布&#xff0c;使得相似的对象有更高的概率被选择&#xff0c;而不相似的对象有较低的概率被选择。 　 &#xff12;) SN…
阅读更多...
t-SNE 原理及Python实例

t-SNE 原理及Python实例

由于毕业设计有部分工作需要对比两个图像数据集合的差异&#xff0c;为了可视化差异&#xff0c;利用了目前降维首选的t-SNE。我花了点时间看了sklearn里面关于这部分的文档&#xff0c;也查阅了相关博客&#xff0c;最终成功的将两种图片数据集作了一个可视化的对比。我觉得这…
阅读更多...
t-SNE算法解析与简单代码实现

t-SNE算法解析与简单代码实现

t-SNE算法解析与简单代码实现 t-SNESNE基本原理和介绍SNE原理推导t-SNE的引入Symmetric SNE拥挤现象关于 σ \sigma σ的求法 代码解析参数说明 Reference t-SNE t-SNE感觉就是将两个数据点的相似度转换为实际距离的算法 t-SNE(t-distributed stochastic neighbor embedding)是…
阅读更多...
t-SNE

t-SNE

t-SNE 文章目录 t-SNE原理SNE(Stochastic Neighbor Embedding)t-SNE对称SNE拥挤问题不匹配的尾部可以补偿不匹配的维度 sklearn.manifold.TSNE参数返回对象的属性Methods 附录Kullback-Leibler divergencest-distributionmanifold learning&#xff08;流形学习&#xff09;Swi…
阅读更多...
【33】t-SNE原理介绍与对手写数字MNIST的可视化结果

【33】t-SNE原理介绍与对手写数字MNIST的可视化结果

如有错误&#xff0c;恳请指出。 这篇博客将会介绍一个无监督的降维算法——t-SNE&#xff0c;其是一个常用的降维可视化工具&#xff0c;下面会记录一下李宏毅老师对其的原理介绍&#xff0c;然后我做了一个实验&#xff0c;用其来对手写数字&#xff08;MNIST数据集&#xff…
阅读更多...
【论文学习之SNE-RoadSeg】跑通SNE-RoadSeg代码

【论文学习之SNE-RoadSeg】跑通SNE-RoadSeg代码

0 序言 作为一个论文学习的小白&#xff0c;第一次去跑一篇论文代码可谓是下了老大功夫。从一开始的陌生&#xff0c;到现在逐渐熟练&#xff0c;对于如何正确跑通论文代码也有了较为清晰的方法步骤。这段时间跟着学长学习研究论文SNE-RoadSeg&#xff0c;所以接下来我将围绕此…
阅读更多...
降维系列之 SNE与t-SNE

降维系列之 SNE与t-SNE

t-SNE是一种经典的降维和可视化方法&#xff0c;是基于SNE&#xff08;Stochastic Neighbor Embedding&#xff0c;随机近邻嵌入&#xff09;做的&#xff0c;要了解t-SNE就要先了解SNE。本文同样既是总结&#xff0c;又是读论文笔记。 SNE 随机近邻嵌入 SNE的的第一步是用条…
阅读更多...
t-SNE算法详解

t-SNE算法详解

前言 此处只作为自己学习理解的笔记之用&#xff0c;转载于https://blog.csdn.net/sinat_20177327/article/details/80298645 t-SNE(t-distributed stochastic neighbor embedding)是用于降维的一种机器学习算法&#xff0c;是由 Laurens van der Maaten 和 Geoffrey Hinton在…
阅读更多...
t-SNE数据降维可视化

t-SNE数据降维可视化

t-SNE数据降维可视化 – 潘登同学的Machine Learning笔记 文章目录 t-SNE数据降维可视化 -- 潘登同学的Machine Learning笔记 t-SNE的基本思想SNE(Stochastic Neighbor Embedding)SNE的主要缺点距离不对称存在拥挤现象 如何确定 σ \sigma σ总结t-sne代码实现 对比t-sne与UMAP…
阅读更多...
【机器学习】基于t-SNE数据可视化工程

【机器学习】基于t-SNE数据可视化工程

一、说明 t-SNE (t-Distributed Stochastic Neighbor Embedding)是一种常用的非线性降维技术。它可以将高维数据映射到一个低维空间（通常是2D或3D）来便于可视化。Scikit-learn API提供TSNE类，以使用T-SNE方法可视化数据。在本教程中，我们将简要学习如何在 Python 中使用 TS…
阅读更多...
推荐文章

Copyright @ 2022~2023