• Home
  • Services
    • DBA Support
    • DBA Consultancy Services
    • PostgreSQL Support
    • Website Maintenance
  • Courses

    About Courses

    • List Of Courses
    • Become an Instructor
    Greenplum Database

    Greenplum Database

    $2,000.00 $1,500.00
    Read More
  • Company
    • FAQs
    • About Us
    • Contact
  • Events
  • Portfolio
  • Blogs
    • Blog – RayaFeeL
    • Blog – PostgreSQL Support
    • Blog – PostgreSQL Migration
    • Blog – All DB’s
    • Blog – Linux
    • Blog – Medical Coding
      • Cart

        0

    Have any question?
    (+91)8838953252
    ITsupport@rayafeel.com
    RegisterLogin
    RayaFeeL
    • Home
    • Services
      • DBA Support
      • DBA Consultancy Services
      • PostgreSQL Support
      • Website Maintenance
    • Courses

      About Courses

      • List Of Courses
      • Become an Instructor
      Greenplum Database

      Greenplum Database

      $2,000.00 $1,500.00
      Read More
    • Company
      • FAQs
      • About Us
      • Contact
    • Events
    • Portfolio
    • Blogs
      • Blog – RayaFeeL
      • Blog – PostgreSQL Support
      • Blog – PostgreSQL Migration
      • Blog – All DB’s
      • Blog – Linux
      • Blog – Medical Coding
        • Cart

          0

      Blog

      • Home
      • Blog
      • Blog
      • While Importing CSV files with text fields that contain double quotes using postgres COPY

      While Importing CSV files with text fields that contain double quotes using postgres COPY

      • Posted by 2ndnijam
      • Categories Blog
      • Date February 10, 2019
      • Comments 0 comment

      While Importing CSV files with text fields that contain double quotes using postgres COPY .
      By following steps you can neglect the double quotes at the time of using postgres COPY restoration

      1. connecting windows server

      D:
      cd D:\PostgresPlus\9.4AS\bin
      psql -U enterprisedb -d Health_DC

      2. Sample Data’s :

      "city_name"|"pincode"|"taluka_name"|"district_name"|"state_name"
      "Arong"|"744301"|"Car Nicobar"|"Nicobar"|"Andaman and Nicobar Islands"
      "Sawai"|"744301"|"Car Nicobar"|"Nicobar"|"Andaman and Nicobar Islands"
      "Chuckchucha"|"744301"|"Car Nicobar"|"Nicobar"|"Andaman and Nicobar Islands"
      "Kinyuka"|"744301"|"Car Nicobar"|"Nicobar"|"Andaman and Nicobar Islands"
      "Big Lapati"|"744301"|"Car Nicobar"|"Nicobar"|"Andaman and Nicobar Islands"
      "Perka"|"744301"|"Car Nicobar"|"Nicobar"|"Andaman and Nicobar Islands"

      3. Following copy command will neglect the double quote

      2ndquadrant.in=# COPY ehis.citytemp FROM 'D:\2ndq/nijamutheen.csv' DELIMITER '|' CSV HEADER;
      ERROR: character with byte sequence 0x9d in encoding "WIN1252" has no equivalent in encoding "UTF8"
      CONTEXT: COPY citytemp, line 358646

      You need to change the character set on your windows command line before running the script with chcp. Or in postgresql you can:

      SET CLIENT_ENCODING TO 'utf8';

      Before importing the file.

      2ndquadrant.in=# SET CLIENT_ENCODING TO 'utf8';
      SET
      
      2ndquadrant.in=# COPY ehis.citytemp FROM 'D:\2ndq/nijamutheen.csv' DELIMITER '|' CSV HEADER;
      COPY 679249

       

      If  your excel sheet not contain with column name means like ‘cityname’, ‘pincode’..etc means your copy command should be below format

      Example : citytemp excel contain below format data’s means COPY Format should be  neglect HEADER

      "Arong"|"744301"|"Car Nicobar"|"Nicobar"|"Andaman and Nicobar Islands"
      "Sawai"|"744301"|"Car Nicobar"|"Nicobar"|"Andaman and Nicobar Islands"
      "Chuckchucha"|"744301"|"Car Nicobar"|"Nicobar"|"Andaman and Nicobar Islands"
      "Kinyuka"|"744301"|"Car Nicobar"|"Nicobar"|"Andaman and Nicobar Islands"
      "Big Lapati"|"744301"|"Car Nicobar"|"Nicobar"|"Andaman and Nicobar Islands"
      "Perka"|"744301"|"Car Nicobar"|"Nicobar"|"Andaman and Nicobar Islands"

      COPY FROM for above data format 

      COPY ehis.citytemp FROM 'D:\2ndq/nijamutheen.csv' DELIMITER '|' CSV;

      For Example,

      If some column not contains data means you have to use NULL values while using COPY FROM

      id,name
      1,nijam
      2,karthick
         ,nijam
      4 ,   
      5,

      So while copying the values you have to mention value oof null otherwise copy command will throw the error

      COPY schema.table FROM 'D:\2ndquadrant\nijam.csv' delimiter ','  CSV header NULL AS 'null';

      If you want to insert system date While postgres COPY, for this purpose you have to use now() function
      for example data should be below format

      date,district
      now(),chennai

      POSTGRES SEQUENCE UPDATED :
      You might have noticed that after bulk inserting records using the COPY statement in PostgreSQL the sequence IDs are not getting updated for any further inserts later on, and it would throw duplicate sequence ID errors.
      So you would be wondering what makes this COPY statement different that it does not update the sequences,Don’t forget to update the sequence in PostgreSQL after a COPY command.

      Use Below anyone of command to update the postgres sequence after bulk insertion or after COPY TO command.

      ALTER SEQUENCE ehis.citytemp_seq RESTART WITH 649328;
      SELECT setval('ehis.citytemp_seq', 649328);
      SELECT setval('ehis.citytemp_seq', 649328);

      POSTGRES MATERIALIZED VIEW REFRESHMENT :
      –Below could block other connections which are trying to read from the materialized view. this is the preferbale in production environment.

      REFRESH MATERIALIZED VIEW CONCURRENTLY ehis.citytemp ;

      — Below materilaized view Refreshment both are same , WITH DATA is default

      refresh materialized view CONCURRENTLY ehis.citytemp with data;
      
      refresh materialized view CONCURRENTLY ehis.citytemp;

      — Refresh the bulk materialized view on particular schema

      select 'REFRESH MATERIALIZED VIEW '|| schemaname||'.'||matviewname ||';' as mview from pg_matviews where schemaname ='ehis';

      –To finding the materialized view whose name start with ‘city’

      select * from pg_matviews where schemaname ='ehis' and matviewname like 'city%' order by matviewname;

      –To finding the materialized view whose name with ‘city’

      select * from pg_matviews where schemaname ='ehis' and matviewname like '%city%' order by matviewname;

      • Share:
      2ndnijam

      Previous post

      Easiest way to fix postgres data partition full without any data loss ?
      February 10, 2019

      Next post

      How to Compare both table values in postgres ?
      February 12, 2019

      Leave A Reply Cancel reply

      You must be logged in to post a comment.

      Login with:

      Login with Google Login with Twitter Login with LinkedIn Login with Microsoft


      Search

      ADVERTISEMENT

      Latest Posts

      PostgreSQL Patching version 9, 10,11
      10Oct2019
      Tools for PostgreSQL
      16Sep2019
      Postgres user creation and restrict DDL & database access
      13Sep2019
      PostgreSQL SSL Setup
      07Sep2019
      How to DELETE current XLOG / WAL LOG in postgresql database ?
      19Aug2019

      Latest Courses

      PostgreSQL Database

      PostgreSQL Database

      $600.00 $500.00
      Greenplum Database

      Greenplum Database

      $2,000.00 $1,500.00

      Preview Course

      Free

      Recent Forum Topics

      • thought behind whiteboard activity
      • Are you going to take your first ste
      • How to start working on an application?
      • please let me know pre requirements to increase work_mem
      • how to copy some data in one table to another table in postgres

      2ndquadrant.in

      (+91) 8838953252

      ITsupport@rayafeel.com

      Company

      • About Us
      • Contact
      • Our Team
      • Blog

      COURSES

      • List Of Course
      • Become An Instructor
      • Events
      • Postgres Support Blog

      Support

      • DBA Support
      • Consultancy Services
      • Postgres Migration Blogs
      • Forum

      Recommend

      • Groups
      • Login
      • FAQs
      • SignUp

      IT Services by rayafeel.com. Powered by Rayafeel Technologies Pvt Ltd.

      • Privacy
      • Terms

      Become An Instructor?

      Join thousand of instructors and earn money hassle free!

      Get Started Now

      Login with:

      Login with Google Login with Twitter Login with LinkedIn Login with Microsoft

      Login with your site account

      Lost your password?

      Not a member yet? Register now

      Register a new account

      Are you a member? Login now