Golden Arc

BERT基础

Transformer基础General Structure： 词向量编码+位置编码 -> 6 * Encoder -> 6 * Decoder -> 下游输出 Encoder：输入部分： 词嵌入input embedding：将词映射到高维向量表示 位置编码positional encoding：加性位置编码+三角函数： 偶数位置用sin，奇数位置用cos 为什么引入位置编码： Transformer的并行化处理导致单词的上下文顺序信息不明显（可以任意调换位置） 解决RNN类网络的连乘效应：远距离梯度消失，近距离梯度主导 Embedding + 位置编码作为Transformer的输入 绝对位置信息中包含了相对位置信息，pos+k位置的位置信息可以由pos位置和k位置的位置信息的线性组合形式表示（三角函数性质） 注意力机制： 基本架构： Q,K,V的获取方式： 用原先的Embedding+位置编码的词向量，分别乘以Wq，Wk，Wv。（W矩阵是学习目标，经过随机初始化再迭代更新） 除以dk的原因：QK相乘的值很大，经过softmax在反向传播中 ...

网络基本结构：七层网络模型（OSI）及其协议：应用层： 直接面向用户和应用；通常有自定义的个性化协议 协议： HTTP超文本传输协议：Web浏览器和Web服务器之间传输的协议。端口号80 HTTPS超文本传输加密协议：加密数据传输，确保传输安全和可靠。使用SSL/TLS认证。端口号443 FTP文件传输协议：客户端和服务端之间的文件传输，支持用户认证和访问控制。端口号21 SMTP简单邮件传输协议：发送电子邮件的协议，支持多种编码形式。端口号25 POP3邮局协议版本3：从邮件服务器上下载信息的标准协议，支持用户认证和访问控制。端口号110 IMAP4互联网消息访问协议版本4：从邮件服务器上下载邮件的标准协议，支持在线操作和多种访问。端口号143 DNS域名系统：将域名转换为IP地址的协议。端口号53 DHCP动态主机配置协议：用于自动分配IP地址、子网掩码、网关等网络参数的协议。端口号67和68 SSH安全外壳协议：加密和保护远程登录会话及其他网络服务，支持用户认证和加密传输数据。端口号22 Telnet远程登录协议：在客户端和服务器之间进行远程登录和命令行操作，不支持加密 ...

开源大模型 Grok-1基本参数： Parameters: 314B Architecture: Mixture of 8 Experts (MoE) 混合专家模型 Experts Utilization: 2 experts used per token Layers: 64 Attention Heads: 48 for queries, 8 for keys/values Embedding Size: 6,144 Tokenization: SentencePiece tokenizer with 131,072 tokens Additional Features: Rotary embeddings (RoPE) Supports activation sharding and 8-bit quantization Maximum Sequence Length (context): 8,192 tokens 项目架构： run.py -> runner.py -> checkpoint.py -> model.py run.py(控制参数 ...

Self-Tracking to Do LessAuthorSarah Homewood, CHI 2023 KeywordsSelf-Tracking, Phenomenology, Long COVID, COVID 19, Fitbit, Heart-rate monitor, Step counting, Post COVID-19 syndrome, pacing technologies, autoethnography, fitness tracking technologies WHAT autoethnography of long COVID, using Fitbit WHY the extraordinary experience of living with a new illness in line with a recent shift within the feld of HCI towards frst-person methodologies and phenomenological approaches HOW Autoethnograph ...

Five Provocations for Ethical HCI ResearchAuthorBarry Brown, Alexandra Weilenmann, Donald McMillan, Airi Lampinen, CHI 2016 KeywordsEthics; Human Trials; Research Practice WHAT to question some of the taken for granted foundations of ethics in HCI WHY HCI research projects have generated controversy due to their purported unethicality ethical creep: virtuous but impractical positions are advocated, and little attention is paid to how seemingly ethical positions can delay, damage or stop resear ...

Towards a Bedder Future: A Study of Using Virtual Reality while Lying DownAuthorThomas van Gemert, Kasper Hornbæk, Jarrod Knibbe, Joanna Bergström, CHI 2023 Keywordsvirtual reality, supine, bed, room-scale, lying down, movement, user experience WHAT developed a custom driver for SteamVR to transform the virtual coordinate space 3 topics about using VR in bed how everyday movements become tricky when lying down users can use the applications while lying down comfortably discuss how the illusi ...

Trash in Motion: Emergent Interactions with Robotic TrashcansAuthorBarry Brown, Fanjun Bu, Ilan Mandel, Wendy Ju, CHI 2024 KeywordsPublic interaction, human-robot interaction, ethnomethodology WHAT How human without prior training or information about their use interactes with the robotic trashcans? WHY autonomous robots become common in use in the city life little or no training information given about their use and behavior HOW Methods Wizard of Oz Apparatus 2 robotic trashcans with no ...

Frustration: Still a Common User ExperienceAuthorMORTEN HERTZUM, KASPER HORNBÆK, ACM Transactions on Computer-Human Interaction 2023 KeywordsFrustration, time lost, computer self-efficacy, user experience WHAT A web-based logging tool collects frustrations from each of the 234 users during one hour of computer use WHY only one replication much has changed over the past fifteen years regarding technology and its use Too many reports inflates the frequency of frustrations Participants limitati ...

Is Time on Smartphones Well Spent?AuthorMaria A. Rasmussen, Julie O. Frydendahl, Elisa D. Mekler, Kasper Hornbæk Keywordssmartphones; user experience; well-being WHAT studies on the time spent on smartphones, the specific applications used, the motivation for using them and their effects on well-being WHY limitations of previous studies no explanation about how the applications that the participants rated were selected no definition and measurement about how being happy or unhappy with an app ...