Regularize, cleanup, and refactor various AWS features #3229
+276
−173
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Provide macro constants for all sources for AWS key values. The possible sources and prefix names are as follows: | Source | Macro Prefix | |--------------------|--------------| | getenv() Variables | AWS_ENV_XXX | | .rc file keys | AWS_RC_XXX | | .aws profiles | AWS_PROF_XXX| | URL fragment keys | AWS_FRAG_XXX| * The macros are in include/ncs3sdk.h. * Some previous macros in other files have been removed.
Provide macro constants for all sources for AWS key values. The possible sources and prefix names are as follows: | Source | Macro Prefix | |--------------------|--------------| | getenv() Variables | AWS_ENV_XXX | | .rc file keys | AWS_RC_XXX | | .aws profiles | AWS_PROF_XXX| | URL fragment keys | AWS_FRAG_XXX| * The macros are in include/ncs3sdk.h. * Some previous macros in other files have been removed.
…tcdf-c into s3global.tmp
Begin the consolidation of global state into two files: libdispatch/dglobal.c and include/ncglobal.h.
## Primary Changes
* New files: include/ncglobal.h and libdispatch/dglobal.c.
* Refactor NCglobalstate decls from nc4internal.h to ncglobal.h.
* Refactor NCglobalstate implementation from nc4internal.c and ddispatch.c to dglobal.c.
* Replace/augment ``#include "nc4internal.h"`` with ``#include "ncglobal.h"``.
* Refactor ChunkCache implementation from nc4internal.c to dglobal.c.
* Side effect: clear chunkcache using memset.
## Secondary Changes
* Modify ncs3sdk.h to define new macros for a variety of AWS keys for environment variables, .aws/config profile keys, AWS-related .ncrc keys, and AWS URI fragment keys.
* Side effect: replace old macros and in-line string constants with the new macros. Changes primarily in libdispatch/ds3util.c and libdispatch/ncs3sdk_h5.c.
* Fix a problem in some tests in nczarr_test: modify cp, unzip and gunzip to remove target before overwriting.
* Fix minor bug in libdap4/d4meta.c
* Fix minor bug in plugins/H5Zshuffle.c.
* Modify nclog to ensure that error log messages always get printed irrespective of the logging state.
<!-- Required Branch Pre-cursors 1. s3forward.dmh 2. ncdumpfixes.dmh 3. s3global.dmh --> ## Primary Change(s) * Refactor to move the AWS constants and code to new files: daws.c and ncaws.h. * Provide macro constants for all sources for AWS key values; the possible sources and prefix names are as follows: | Source | Macro Prefix | |--------------------|--------------| | .rc file keys | AWS_RC_XXX | | .aws profiles | AWS_PROF_XXX | | getenv() Variables | AWS_ENV_XXX | | URL fragment keys | AWS_FRAG_XXX | | URL note keys | AWS_NOTE_XXX | Notes: * The macros are in include/ncaws.h * Some previous macros in other files have been removed. ## Secondary changes 1. Convert NCglobalstate.chunkcache to struct from previous pointer to struct. This entails a number of small changes to handle the change.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Regularize, cleanup, and refactor various AWS features
Ready for review.
Primary Change(s)
Notes:
Secondary changes